Private interconnect oracle 10g RAC configuration

Can you please answer below questions?
Why does a private interconnect need a switch, and why is straight through cables not supported.

Hi,
Why do you need a switch between the nodes".When network plugs are pulled out from one node on a two node cluster, a split brain scenerio occurs. (just it's enough)
If you are using a crossover cable and you shutdown node (A) you will loose the network (private) link from node (B) (this happens in some servers.), Oracle RAC will not work with private network link down, both nodes will down and will not start until you get link on network private. (Goodbye high availability)
You will be very unhappy with the error ORA-29740. Prelude to suggest this note: Troubleshooting ORA-29740 in a RAC Environment [ID 219361.1]
There is no "why" Oracle RAC does not support crossover cable because RAC depends of SWITCH (it's a Hardware Requirements) and any problems in your environment Oracle will force you to implement a supported solution.
You will have implemented a poor environment, if you not use a GB switch for private network.
Oracle Words:
*Physical Layout of the Private Interconnect*
The basic requirements are described in the Installation Guide for each platform. Additional information about certification can be found on Metalink Certify.
The interconnect as identified by both subnet number and interface name must be configured on all clustered nodes.
*A switch between the clustered nodes is an absolute requirement.*
*Cluster Interconnect in Oracle 10g and 11g [ID 787420.1]*
Regards,
Levi Pereira

Similar Messages

Link Based IPMP Private Interconnect - Oracle 10g

I am configuring IPMP for Oracle 10g RAC private and Private Interconnect.
Following an Oracle white paper we are configuring LINK Based IPMP in an Active/Active Configuration
I have information from metalink for configuring the VIP, however I am a little unsure about the configuration of the private interconnect.
Is it just a case of removing the cluster interconnect from oifcfg and using cluster_interconnect for ASM and Database instances.
Is there any other considerations for Oracle 10g RAC deployment beside making change using oifcfg as well as using private_interconnect initialization parameter.
Thanks
Paul

The following notes are of help for you:
How to Setup IPMP as Cluster Interconnect (Doc ID 368464.1)
Configuring Solaris IP Multipathing (IPMP) for the Oracle 10g VIP (Doc ID 283107.1)
White paper on IPMP:
http://www.oracle.com/technetwork/articles/systems-hardware-architecture/ha-rac-networking-ipmp-168440.pdf
Raj Mareddi
http://www.freeoraclehelp.com

Oracle 10g RAC - Private Interconnect on Private non-routable VLAN

In our data center there is an existing Oracle 10g RAC configured with private VLAN for Interconnect administered by a different group of DBAs.
We are designing a new, separate Oracle 10g RAC environment to support our application.
When we discussed with our data center folks to set up a private VLAN for our RAC Interconnect, they suggest to use the same existing Private VLAN used by other Oracle RAC configurations. In that case the Interconnect IPs will be on the same subnet as other Oracle RAC configurations.
For example, if
RAC1 with 2 nodes is using 192.168.1.1 and 192.168.1.2 in the VLAN_1 for the Interconect, they want us to use the same VLAN_1 with Interconnect IPs 192.168.1.3 and 192.168.1.4 for our 2 node RAC.
Is Sharing same subnet on the same Private VLANs for interconnects of different RAC configurations supported?
Will that cause any performance hit? That means the Interconnect IPs of One RAC configuration is pingable from other RAC configuration.
Did anyone come across such a design?
Could not find any info on this on Metalink.
Thanks

yes
this is practically very much feasible.. as you would have only 4 m/c in ip subnet .... and this is very much less than the public subnet which we should refrain from using from interconnect.

Shared Disk Subsystem for Oracle 10g RAC

I am investigating the feasibility of clustering two 220Rs together in an Oracle 10g RAC configuration. Can I use a D1000 disk array for the shared storage between the two nodes? Can you briefly describe how to do it or point me to documentation on how to do it?

hi dstmark,
can you kindly share some information with me? i am thinking of building up a small lab at home using A1000 to build up an Oracle RAC, as a1000 have 2 scsi port and have a build in controller.
but i am unsure about the detail software requirement, as google return some conflict information.
is SUN cluster would be enough? do i need to get something like Veritas Cluster? which may require a license key....
thanks for sharing information.

How to configure Oracle 10g RAC on windown sever 2003

hi all
plz tell me
how to configure Oracle 10g RAC on windown sever 2003 can any body help me , give be any link
plz it is very necessary for me
Regards

Hello,
There are a good doc written by Philip Newlan at www.jobcestbon.com/oracle/RacOnWindows.pdf
Regards,
Rodrigo Mufalani
http://mufalani.blogspot.com

Time synchoronization while configuring Oracle 10g RAC

I am installing Oracle 10g RAC on 3 EHEL5.0 systems. The problem is the time can not be synchoronized although I had already enaled the Network Time Protocol and let all the related servers to be synchoronized with the same NTP server.
I want to know why the time cannot be synchoronized on different servers?

Time synchronization with RAC is critical. Oracle will refuse to install if you do not perform this step.
You have three choices. The more risky one is to configure access to an external time server. I don't like this because it does create a path between my data server and the internet. The second is to build a time server in our data center that you reference. This is the best option as you can, and should, use that server for synchronize everything which is a great help with replication. The third choice is to arbitrarily choose one node and synchronize the other nodes with it. The downside is that if you lose that node you are at risk.
There are plenty of instructions on how to create a time server on the internet but no matter which choice you make ... you will need root privs on operating system to configure it.

Oracle 10G RAC - Public & Heartbeat on 1 NIC

Hello all,
actually I am installing Oracle 10G RAC on rhel 4 (4 node cluster). But the Cluster Verification Utility aborts with errors. I checked the configToolAllCommands and tried to run the failed commands manually:
#/opt/oracle/crs/bin/oifcfg setif -global eth0.100/172.18.0.0:cluster_interconnect eth0.728/172.16.128.0:public eth0.498/172.17.1.0:cluster_interconnect
PRIF-50: duplicate interface is given in the input
PRIF-50: duplicate interface is given in the input
PRIF-50: duplicate interface is given in the input
Question:
Is it possible to put Public & Heartbeat on one NIC (eth0.728 & eth0.498)
If not is their any workaround for that issue?
Output /etc/hosts
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
172.18.253.48 eu0266.[company].net cfmaster
172.16.128.11 eu0200.[company].net eu0200
172.16.128.12 eu0201.[company].net eu0201
172.16.128.13 eu0202.[company].net eu0202
172.16.128.14 eu0203.[company].net eu0203
172.18.13.11 eu0200m.[company].net eu0200m
172.18.13.12 eu0201m.[company].net eu0201m
172.18.13.13 eu0202m.[company].net eu0202m
172.18.13.14 eu0203m.[company].net eu0203m
# Private section
172.17.1.11 eu0200-priv.[company].net eu0200-priv
172.17.1.12 eu0201-priv.[company].net eu0201-priv
172.17.1.13 eu0202-priv.[company].net eu0202-priv
172.17.1.14 eu0203-priv.[company].net eu0203-priv
# Virtual section
172.16.128.16 eu0200-vip.[company].net eu0200-vip
172.16.128.17 eu0201-vip.l[company].net eu0201-vip
172.16.128.18 eu0202-vip.[company].net eu0202-vip
172.16.128.19 eu0203-vip.[company].net eu0203-vip
Output install log:
Checking existence of VIP node application (required)
Check failed.
Check failed on nodes:
eu0203,eu0202,eu0201,eu0200
Checking existence of ONS node application (optional)
Check ignored.
Checking existence of GSD node application (optional)
Check ignored.
Post-check for cluster services setup was unsuccessful on all the nodes.
Command = /opt/[company]/oracle/crs/bin/cluvfy has failed
INFO: Configuration assistant "Oracle Cluster Verification Utility" failed
*** Starting OUICA ***
Oracle Home set to /opt/[company]/oracle/crs
Configuration directory is set to /opt/[company]/oracle/crs/cfgtoollogs. All xml files under the directory will be processed
INFO: The "/opt/[company]/oracle/crs/cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same.
SEVERE: OUI-25031:Some of the configuration assistants failed. It is strongly recommended that you retry the configuration assistants at this time. Not successfully running any "Recommended" assistants means your system will not be correctly configured.
1. Check the Details panel on the Configuration Assistant Screen to see the errors resulting in the failures.
2. Fix the errors causing these failures.
3. Select the failed assistants and click the 'Retry' button to retry them.
Thanks in advance for your help!
Regards

Tads wrote:
Hi all,
I have a question about Oracle RAC and network interface.
We're using Oracle 10gR2 RAC with two nodes on Linux Red Hat.
Let's assume that the public network interface goes down.
I would like to know what happens with existing connections
on node with network interface with problems.
Are connections frozen, actives?
Can the users continue to use theses existing connections using the another node of RAC?If the interface is down? what do you think? All connections to this node will die. How does your application handle fail-over, does it attempt to reconnect or just have a complete application failure?
You should spend some time in a test lab where you can test this stuff for yourself. Read the documentation and there are tons of sites out there that purport to have all of your RAC/TAF/FAN/FAF questions. - I would read and trust the documentation first.
>
I know that the listener goes down and any other connections is allowed.
Thank you very much!!!!

Oracle 10g RAC on Windows

Installed Oracle 10g RAC on Windows... Everything was
working fine, now when I install CRS it completed at 100% and then in the configuration it gives error
INFO: exitonly tools to be excuted passed: 0
INFO: Starting to execute configuration assistants
INFO: Command = C:\WINDOWS\system32\cmd /c call C:\oracle\product\10.2.0\crs/install/crssetup.config.bat
PROT-1: Failed to initialize ocrconfig
Step 1: checking status of CRS cluster
Step 2: creating directories (C:\oracle\product\10.2.0\crs)
Step 3: configuring OCR repository
ocr upgrade failed with (-1)
Execution of the plugin was aborted
INFO: Configuration assistant "Oracle Clusterware Configuration Assistant" was canceled.
*** Starting OUICA ***
Oracle Home set to C:\oracle\product\10.2.0\crs
Configuration directory is set to C:\oracle\product\10.2.0\crs\cfgtoollogs. All xml files under the directory will be processed
INFO: The "C:\oracle\product\10.2.0\crs\cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same.
I am using VMware WorkStation 5
http://www.oracle-base.com/articles/10g/OracleDB10gR2RACInstallationOnWindows2003UsingVMware.php
I can see my disks from both machines. Open the "Computer Management" dialog (Start > All Programs > Administrative Tools > Computer Management
I use Windows XP SP2 as the host operating system
Any suggestion?
Message was edited by:
MEXMAN

You know its not certified?
My first thought when I read the subject of this thread, and before reading what you posted, is that you've either done no research on RAC or didn't understand what you've read.
You can create a RAC cluster using any operating system certified by Oracle.
But the operating system is the least important part of creating a cluster. The questions you need to be able to address are:
1. What is your solution to created shared storage?
2. What is your solution to create the cache fusion interconnect and VIPs?
If you don't have an answer to these questions you can not build a RAC cluster.

Problem with Oracle 10g RAC VIP network setting at Solaris 9

Dear All,
I have tried to set up a Oracle 10g RAC Release 2.
With OS solaris 9, and 2 nodes.
The nodes setting as the following:
nodes 1:
Public address: 172.16.0.121
Private address: 192.168.0.121, 192.168.1.121 (dual path for heart beat)
nodes 2:
Public address: 172.16.0.122
Private address: 192.168.0.122, 192.168.1.122
And i have assigned two IP adress 172.16.0.131, 172.16.0.132 as the VIP address for the
RAC.
And the following is the /etc/hosts file:
root@shk01 # cat /etc/hosts
# Internet host table
127.0.0.1 localhost
# public address
172.16.0.121 shk01 loghost
172.16.0.122 shk02
# heart beat
192.168.0.121 shk01-priv1
192.168.1.121 shk01-priv2
192.168.0.122 shk02-priv1
192.168.1.122 shk02-priv2
# VIP address
172.16.1.131 vip-shk01
172.16.1.132 vip-shk02
But when i run the command "$ ./runcluvfy.sh comp nodecon -n shk01,shk02 -verbose"
it shows the error:
ERROR:
Could not find a suitable set of interfaces for VIPs.
Result: Node connectivity check failed.
I did try to add the VIP address on bge0:1, as i using bge0 for the public address.
Both nodes i did using the same interface name for it.
Anyone have idea for me to check out the error?
Also, I have other question about the raw device.
As there is option for setting for ASM or raw device. If choosing raw device, does it mean that it just need to
format the storage disk but without newfs it? And then the Oracle program will able to handle it?
Thanks,
Xentar

You don't seem to state categorically that you are using Solaris Cluster, so I'll assume it since this is mainly a forum about Solaris Cluster (and IMHO, Solaris Cluster with Clusterware is better than Clusterware on its own).
Clusterware has to see the same device names from all cluster nodes. This is why Solaris Cluster (SC) is a positive benefit over Clusterware because SC provides an automatically managed, consistent name space. Clusterware on its own forces you to manage either the symbolic links (or worse mknods) to create a consistent namespace!
So, given the SC consistent namespace you simple add the raw devices into the ASM configuration, i.e. /dev/did/rdsk/dXsY. If you are using Solaris Volume Manager, you would use /dev/md/<setname>/rdsk/dXXX and if you were using CVM/VxVM you would use /dev/vx/rdsk/<dg_name>/<dev_name>.
Of course, if you genuinely are using Clusterware on its own, then you have somewhat of a management issue! ... time to think about installing SC?
Tim
---

Need to change IP address and host name on a ORACLE 10g RAC setup.

Hi Forum,
I have a task to change the IP address, host name and host related details for the ORACLE 10g RAC 2 node setup. So Can anyone tell me the procedure to do the same on both the nodes.
Regards
Prakash

change IP Address Public + VIP + Interconnect
http://surachartopun.com/2007/01/i-want-to-change-ip-address-on-oracle.html
change VIP name
http://surachartopun.com/2009/01/change-vip-hostname-on-oracle-rac.html
If you'd like to change HOSTNAME... Metalink (NOTE:220970.1)
Can I change the public hostname in my Oracle Database 10g Cluster using Oracle Clusterware?
Hostname changes are not supported in Oracle Clusterware (CRS), unless you want to perform a deletenode followed by a new addnode operation.
The hostname is used to store among other things the flag files and Oracle Clusterware stack will not start if hostname is changed.
Changing the Private Interconnect
o Change of the IP Address: change the IP address in the hosts file and/or DNS and make sure that ASM and the database also use the same interconnect
o Change of the private node name used by Oracle Clusterware: requires a reinstall of Oracle Clusterware
So, Public or Private hostnames can only be changed by removing/adding nodes, or reinstalling Oracle Clusterware. VIP Hostnames can be changed
But you can read this idea (reinstall CRS)
http://www.pythian.com/news/482/changing-hostnames-in-oracle-rac
http://surachartopun.com/2008/12/change-hostnames-oracle-rac.html
Edited by: Surachart Opun (HunterX) on Oct 25, 2009 10:58 PM

Oracle 10g RAC installation on HP-UX 11.31 for SAP ERP 6.04

Dear experts,
We are trying to install SAP ERP 6.0 EHP4 with Oracle 10g RAC on HP-UX 11.31. Please note that, We are using VERITAS cluster filesystem (CFS) for this purpose and not HP-UX ServiceGuard ClusterFileSystem.
*As per SAP procedure, we have installaed plain plain SAP system with single instance Oracle (ref: Configuration of SAP NetWeaver for Oracle Database 10gRAC guide). Now we are first trying to install Oracle Clusterware (CRS) and then we will install Oracle RAC software. Is this procedure right?*
Which file (and its path) we have to use for CRS installation and its patch? Is it runInstaller under /oracle/stage/102_64/clusterware? With this file we can install CRS 10.2.0.1. Similarly for Oracle RAC, which files we are supposed to use, i.e for installation and patch upgrade?
Also can we use clusterware package available from Oracle directly in the SAP environment?
We tried to install CRS with runInstaller while running Configuration Assistant after root.sh script.
The following commands are failing in Configuration Assistant:
/oracle/CRS/102_64/bin/racgons add_config erpprdd1:4948 erpprdd2:4948
/oracle/CRS/102_64/bin/oifcfg setif -global lan1/20.20.20.0:cluster_interconnect lan0/192.168.3.0:public
/oracle/CRS/102_64/bin/cluvfy stage -post crsinst -n erpprdd1,erpprdd2
1st command does not give any output if run manually.
2nd commands o/p: PRIF-12: failed to initialize cluster support services
3rd commands fails in every check
Please suggest the solution and expecting answers to the above mentioned questions.
Thanks & Regards,
Tejas

>
Charles Yu wrote:
> Q1: Oracle RAC with 9.2.x on HP-UX?
> A: For HA environment, cluster software is: MC/SG on HP-UX 11.31; there are optional components of MC/SG for supporting Oracle RAC and SAP application. I was confused that I could not find the installation guide regarding 4.6C on MC/SG HA environment of HP-UX.
> Charles
Relevant docs for Service Guard (SG) cluster are available at http://docs.hp.com. Hope you have checked for the suport of oracle 92x on HP-UX 11.31
>
Charles Yu wrote:
> Q2: Any reason why you don't use a supported database version?
> A: Actually, in order to avoiding the the risk of database upgrade and minimizing the migration risk, top level has decided that keeping the same Oracle version. Indeed, we don't plan the migration of application. On the other hand, it is complicated to do the assessment for application migration.
> Charles
You can go for combined os migration and db release upgrade also at a stretch with the same downtime.

Oracle 10g RAC

Hi All,
I am a research student currently struggling to install Oracle 10g RAC on Windows OS.What i understand about a Oracle RAC database system is that it involves a configuration of multiple hosts or servers joined together with a clustering software and accessing the shared disk structures.What i also understand is that i need a CRS provided by Oracle for a Windows or Linux based RAC installation.
What i don't understand is for a Two node RAC system do i need a Third machine which has the shared storage device.If yes do i have to install a copy of Oracle 10g Enterprise Edition and CRS on the shared storage device moreover do i need to install a copy of Oracle 10g Enterprise edition on each of the nodes or just the clusterware is enough.

Hi Salil
No, you do not need a third machine. You can even setup RAC as a single node on one machine! What you do need is to have some sort of cluster aware file system. Try looking at OCFS on Windows in your case.
If you are interested in a Linux example please have a look at the examples at http://ocpdba.net/oracle9i_rac/cheaprac.html and http://ocpdba.net/oracle10g_rac for an OCFS implementation I am working on.
regards,
Ahbaid

Oracle database not starting up in oracle 10g RAC

Hi!
Recently I came across one problem with one node oracle 10g RAC.When the Oracle database is started,while opening it is giving ORA-03113:End of file on communication channel error.When I saw the the alert trace file and other trace files I found Disk group is exhausted error and it is not able to create .dbf files.Actually it is not a production server and I gave archive log destination in SAN.Even the spfile(content of init_database.ora) is in SAN..
I tried Asmcmd utility to delete the archive log files.As the oracle is not available I am not able to asmcmd prompt.
How to change the destination of archive log and to remove the old archive log files(as it is a testing environment we can remove) from SAN?Please let me know.
Thanks & Regards
Srikanth MVS

keithrust wrote:
On VMware there's a known issue with Oracle databases on a Windows client not starting up properly all the time and a manual startup using oradim -start -sid <whatever> is required to get it fully running. Hmmm, doing it several time, and never seen such issue. Which "known issue" and by who are you talking about ?
I created a brand new Oracle VM Windows 2003 32-bit server, installed the Oracle drivers for paravirtualization, and whammo, the problem is still hereI'm sure, you miss something somewhere in the config. Right now, you're on supported configuration, you could either raise a SR to the support, or get help from your peer on Oracle Database General forum.
Ah, but it's not a Windows issue. On a non-VM Windows box the database starts just fine all the time. Again, this is a known issue acknowledged by Oracle on the VMware side, I'm just surprised it exists on the Oracle VM side.Again, give more details about this "known issue". Never heard about that, eventhough I've been around for years.
I was afraid you were going to ask that. I'll have to search for it again, but I think you can do the same as well....Well, I doubt you could find a Metalink note about Oracle on VMWare. So far, Oracle has always refused to support database on OS virtualized on VMWare (or any VM software other than Oracle VM). Based on that, you could be sure, your "known issue" is not an issue on Oracle VM.
If you want more help, again, give more details about your issue.
Nicolas.

What is acceptable level of Private Interconnect Latency for RAC

We have build 3 Node RAC on RHEL5.4 on VMware.
There is node eviction problem due to loss of Network Heartbeat.
ocssd.log:[ CSSD]2010-03-05 17:48:21.908 [84704144] >TRACE: clssnmReadDskHeartbeat: node 3, vm-lnx-rds1173, has a disk HB, but no network HB, DHB has rcfg 0, wrtcnt, 2, LATS 1185024, lastSeqNo 2, timestamp 1267791501/1961474
Ping statistics from Node2 to Node1 are as below
--- rds1171-priv ping statistics ---
443 packets transmitted, 443 received, 0% packet loss, time 538119ms
rtt min/avg/max/mdev = 0.150/2.030/630.212/29.929 ms
[root@vm-lnx-rds1172 oracle]#
Can this be reason for Node eviction? What is acceptable level of of private interconnect latency for RAC ?

What is acceptable level of of private interconnect latency for RAC ?Normal local network latency should be enough. By the way latency settings are very generous.
Can you check if your to-be-evicted node runs and is reachable when seeing the node eviction messages?
In addition to that: Can you check the log files of the eviced node. Check for time stamps around "2010-03-05 17:48:21.908". Make sure all systems are NTP synchronized.
Ronny Egner
My Blog: http://blog.ronnyegner-consulting.de

Solaris 10 and Hitachi LUN mapping with Oracle 10g RAC and ASM?

Hi all,
I am working on an Oracle 10g RAC and ASM installation with Sun E6900 servers attached to a Hitachi SAN for shared storage with Sun Solaris 10 as the server OS. We are using Oracle 10g Release 2 (10.2.0.3) RAC clusterware
for the clustering software and raw devices for shared storage and Veritas VxFs 4.1 filesystem.
My question is this:
How do I map the raw devices and LUNs on the Hitachi SAN to Solaris 10 OS and Oracle 10g RAC ASM?
I am aware that with an Oracle 10g RAC and ASM instance, one needs to configure the ASM instance initialization parameter file to set the asm_diskstring setting to recognize the LUNs that are presented to the host.
I know that Sun Solaris 10 uses /dev/rdsk/CwTxDySz naming convention at the OS level for disks. However, how would I map this to Oracle 10g ASM settings?
I cannot find this critical piece of information ANYWHERE!!!!
Thanks for your help!

You don't seem to state categorically that you are using Solaris Cluster, so I'll assume it since this is mainly a forum about Solaris Cluster (and IMHO, Solaris Cluster with Clusterware is better than Clusterware on its own).
Clusterware has to see the same device names from all cluster nodes. This is why Solaris Cluster (SC) is a positive benefit over Clusterware because SC provides an automatically managed, consistent name space. Clusterware on its own forces you to manage either the symbolic links (or worse mknods) to create a consistent namespace!
So, given the SC consistent namespace you simple add the raw devices into the ASM configuration, i.e. /dev/did/rdsk/dXsY. If you are using Solaris Volume Manager, you would use /dev/md/<setname>/rdsk/dXXX and if you were using CVM/VxVM you would use /dev/vx/rdsk/<dg_name>/<dev_name>.
Of course, if you genuinely are using Clusterware on its own, then you have somewhat of a management issue! ... time to think about installing SC?
Tim
---

Private interconnect oracle 10g RAC configuration

Similar Messages

Maybe you are looking for