Ora.reco.acfsvol.acfs only on one node on RAC on ODA

We have an ODA (old model) and by a power failure in the data center both boot disks in one node are we gone faulty.
After replacing the chassis, RAID controllers and disks (Oracle Filed Engenieer) reports crsctl stat res -t following:
[grid @ XXXXXXXXA ~] $ crsctl stat res -t
TARGET NAME SERVER STATE STATE_DETAILS
Local Resources
ora.reco.acfsvol.acfs
ONLINE ONLINE XXXXXXXXXA mounted on / cloudfs
OFFLINE OFFLINE XXXXXXXXXBvolume / cloudfs off
is that correct?
Oracle support referred me to MOS 1319263.1, but that's for Exadata ....
Thx
Christoph
(i masked the hostname)

No, this is not correct. Your resource should be online on both nodes.
What happens if you try and start the resource manually using srvctl start filesystem?
Have you checked to see if your volume is online?

Similar Messages

Why RAW partitions are visible only on one node in RAC?

I am having 2 node RAC on Windows 2003 Server.
Can anybody tell me why the RAW partitions are visible on RAC-2 but not on RAC-1.
I Shutted down RAC-2, and I thought all the RAW devices will appear in RAC-1, but it didn't work, it is displaying only LOCAL DRIVE.
Why???

I am using Microsoft Virtual Server and I have configured SCSI type shared disks.
I executed the given command
C:\oracle\product\10.2.0\crs\bin>cluvfy comp ssa -n rac-1,rac-2
Result : Shared storage check was successful on both the nodes..
Both the nodes are working properly but I want to know the reason 'why the shared disks are not visible on node1'?
Thanks
Sushil

Why only gets one node when select many nodes of tree in DWCS4 on Mac OS

I use tag <mm:treecontrol> to create tree in DWCS4 on Mac OS.
When I select many nodes in tree, but I only get one node by method: selectedNodes.
codes of created tree as following:
<mm:treecontrol name='tree' size='20' multiple noheaders>
     <mm:treecolumn state='hidden'>
          <mm:treenode value='A' state='expanded'></mm:treenode>
          <mm:treenode value='B' state='expanded'></mm:treenode>
          <mm:treenode value='C' state='expanded'></mm:treenode>
</mm:treecontrol>
Who can tell me reasons?
Thanks!
comments: if don't use tag <mm:treecolumn>, tree will not show on Mac OS.

Hi macbig,
I finally got to look at my sister's computer. The HDD "Repair Disk" found missing threads, missing directory records, etc. and ended with:
Error: Disk Utility can't repair this disk. Back up as many of your files as possible, reformat the disk, and restore your backed-
up files.
Then, I tried "Verify Disk" and it found invalid volume file count and ended with:
The volume Macintosh HD was found corrupted and needs to be repaired.
Error: This disk needs to be repaired. Click Repair Disk
I guess running Apple Hardware Test is not going to happen. :/
I've ordered online a new 2.5 disk, make a Maverick boot USB, and start from scratch. Do you have any other suggestions?
As for the corrupted old hard drive, do you have any suggestions of how to get out the data somehow?
Thank you so much!

EM DB console works only from one node?

In my 2 node RAC the enterprise manager database console works only from one node i.e the url shows login page only from one node.
http://NODE2:5500/em------>WORKS, SHOWS LOGIN PAGE AND CAN MANAGE ALL INSTANCES FROM HERE
http://NODE1:5500/em------>DOES NOT WORK, DOES NOT SHOW THE LOGIN PAGE.
Please clarify.
Kadhim

I have to guess ... (because you tell neither OS nor database version). Assuming it's 10gR2 or higher, that's expected behaviour, dbconsole runs on one node only, the so-called master node. You can change that, see the documentation or this metalink note:
How to manage DB Control 10.2 for RAC Database with emca
Doc ID: NOTE:395162.1
Werner

Rac One Node on Rac Servers

Hi Xperts
We have this environment:
2 Rac Nodes 11.2.0.3 Enterprise on Oracle Linux 5.9 .
We have one production Database on this Rac and the users ask to create two single instance on each node, something like this:
Node1 -> Rac Prod1, Single Test
Node2-> Rac Prod2, Single Dev
I want to create Rac One node for those Database (Dev, Test) and create New Diskgroups for ech database.
Can I install a Rac One node on those Server with DBCA?
Do I nedd to Install new Database Software ?
Does the installed Rac have some affectation ?
I just want to be sure about this procedure, before to do anything.
Thank you
J.A.

Hi J.A.
Yes you can! However you need to install Grid Infrastructure (GI) in cluster on both nodes, then install database software. Either during software installation or after that, DBCA would allow you to create 1) Single instance, 2) RAC database, 3) RAC One Node database. Keep in mind that RAC One Node is an option (additional license) to the Enterprise Edition of Oracle Database.
I've talked on that topic at the Bulgarian Oracle Users Group at 2011, here is the link to the presentation, you may find it useful. I might upload the videos as well if you need to have something like a proof of concept of just for your own:
http://sve.to/download/1112/
Also I would go with one disk group for both databases, as long as they share the same physical disks I don't see the point of doing that ? Having one diskgroup would allow you to utilize better you disk/space resources.
The procedure would be:
1. Install GI in cluster.
2. Install software libraries.
3. Patch up to 11.2.0.4.
4. Create RAC One Node database using either command line or using Custom Template of DBCA.
At the end of the day, if you have standard edition license you can still install GI in cluster and create single instance databases on each server. The downside of doing that is that you need to manually failover the database to remaining node in case of disaster.
Regards,
Sve

Oracle Binary Currepted in One node 11g RAC

Hi Team,
/oracle(Oracle Home ) folder currepted in one node.
IBM AIX-11.2.0.1
How to reolve the same.
Thanks
Manohar.

1. take the backup of current oracle_home corrupted.
2. Tar the other node oracle_home as below, and copy it to the corrupted node.
tar cvf location <tarname>.tar oraInventory product
3. Extracted the tar on the corrupted node by delting the existing corrupted oracle_home
tar xvf /location/<name>.tar
Now, complete the Oracle RDBMS Cloning Process of the above mentioned untar done in earlier steps:
cd $ORACLE_HOME/clone/bin
Perl clone.pl ORACLE_HOME="<LOCATION>" ORACLE_HOME_NAME="OraDb10g_home1"
Run manual command of “relink all”

"ORA-00928: missing SELECT keyword" only on one instance in RAC

I am using oracle 9i RAC.When I run a query in one instance it is working fine but in another instance of RAC it is giving "ORA-00928: missing SELECT keyword".
I am wondering why it happening in the same RAC.Please suggest me where is the problem.
Thanks

STAR_TRANSFORMATION_ENABLED=true in both instances
My query lools like
SELECT * FROM
     (SELECT coulmn1,col2
               FROM tab1 wh full OUTER JOIN tab2 stg on wh.row_id = stg.row_id
               WHERE wh.MODIFICATION_NUM IS NULL
               OR stg.MODIFICATION_NUM IS NULL
                    OR wh.MODIFICATION_NUM <> stg.MODIFICATION_NUM) WHERE REC_STA <>'D';
Please suggest
Thanks

Ons gsd of one node offline,RAC

crs_start ora.whdb02.ons
Attempting to start `ora.whdb02.ons` on member `whdb02`
Start of `ora.whdb02.ons` on member `whdb02` failed.
whdb01 : CRS-1019: Resource ora.whdb02.ons (application) cannot run on whdb01
CRS-0215: Could not start resource 'ora.whdb02.ons'
oracle@whdb02:/oracle/app/oracle/product/10.2.0/db1/network/admin>$CRS_HOME/bin/onsctl start
ksh: /bin/onsctl: not found.
oracle@whdb02:/oracle/app/oracle/product/10.2.0/db1/network/admin>$ORA_CRS_HOME/bin/onsctl start
clsrons_init failed, stat = 504, ocrerr = 32
clsrons_init failed, stat = 504, ocrerr = 32
onsctl: ons failed to start
oracle@whdb02:/oracle/app/oracle/product/10.2.0/db1/network/admin>
oracle@whdb02:/oracle/app/oracle/product/10.2.0/db1/network/admin>
oracle@whdb02:/oracle/app/oracle/product/10.2.0/db1/network/admin>A_CRS_HOME/bin/onsctl STOP
ksh: A_CRS_HOME/bin/onsctl: not found.
oracle@whdb02:/oracle/app/oracle/product/10.2.0/db1/network/admin>$ORA_CRS_HOME/bin/onsctl stop
onsctl: shutting down ons daemon ...
clsrons_init failed, stat = 504, ocrerr = 32
onsctl: shutdown of ons failed!
oracle@whdb02:/oracle/app/oracle/product/10.2.0/db1/network/admin>$ORA_CRS_HOME/bin/onsctl start
clsrons_init failed, stat = 504, ocrerr = 32
clsrons_init failed, stat = 504, ocrerr = 32
onsctl: ons failed to startoracle@whdb02:/oracle/app/oracle/product/10.2.0/db1/network/ad

gsd log
2010-11-10 13:16:45.515: [    RACG][1] [274628][1][ora.whdb02.gsd]: clsrcexecut: cmd = /oracle/app/oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 540 /oracle/app/oracle/product/10.2.0/crs/bin/gsdctl stat
2010-11-10 13:16:45.515: [    RACG][1] [274628][1][ora.whdb02.gsd]: clsrcexecut: rc = 1, time = 2.703s
2010-11-10 13:16:45.515: [    RACG][1] [274628][1][ora.whdb02.gsd]: end for resource = ora.whdb02.gsd, action = start, status = 1, time = 54.109s
2010-11-10 13:16:45.989: [    RACG][1] [274634][1][ora.whdb02.gsd]: clsrcgetprsrctx: prsr_init_ext returned rc = 3
2010-11-10 13:16:48.694: [    RACG][1] [274634][1][ora.whdb02.gsd]: GSD is not running on the local node
2010-11-10 14:09:27.008: [    RACG][1] [573578][1][ora.whdb02.gsd]: clsrcgetprsrctx: prsr_init_ext returned rc = 3
2010-11-10 14:10:18.016: [    RACG][1] [573578][1][ora.whdb02.gsd]: Failed to start GSD on local node
2010-11-10 14:10:18.016: [    RACG][1] [573578][1][ora.whdb02.gsd]: clsrcexecut: cmd = /oracle/app/oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 540 /oracle/app/oracle/product/10.2.0/crs/bin/gsdctl start
2010-11-10 14:10:18.016: [    RACG][1] [573578][1][ora.whdb02.gsd]: clsrcexecut: rc = 1, time = 51.008s
2010-11-10 14:10:20.720: [    RACG][1] [573578][1][ora.whdb02.gsd]: GSD is not running on the local node
2010-11-10 14:10:20.720: [    RACG][1] [573578][1][ora.whdb02.gsd]: clsrcexecut: cmd = /oracle/app/oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 540 /oracle/app/oracle/product/10.2.0/crs/bin/gsdctl stat
2010-11-10 14:10:20.720: [    RACG][1] [573578][1][ora.whdb02.gsd]: clsrcexecut: rc = 1, time = 2.704s
2010-11-10 14:10:20.720: [    RACG][1] [573578][1][ora.whdb02.gsd]: end for resource = ora.whdb02.gsd, action = start, status = 1, time = 54.109s
2010-11-10 14:10:21.195: [    RACG][1] [573584][1][ora.whdb02.gsd]: clsrcgetprsrctx: prsr_init_ext returned rc = 3
2010-11-10 14:10:23.900: [    RACG][1] [573584][1][ora.whdb02.gsd]: GSD is not running on the local node
2010-11-10 15:33:02.434: [    RACG][1] [618716][1][ora.whdb02.gsd]: clsrcgetprsrctx: prsr_init_ext returned rc = 3
2010-11-10 15:33:56.467: [    RACG][1] [618716][1][ora.whdb02.gsd]: Failed to start GSD on local node
2010-11-10 15:33:56.467: [    RACG][1] [618716][1][ora.whdb02.gsd]: clsrcexecut: cmd = /oracle/app/oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 540 /oracle/app/oracle/product/10.2.0/crs/bin/gsdctl start
2010-11-10 15:33:56.467: [    RACG][1] [618716][1][ora.whdb02.gsd]: clsrcexecut: rc = 1, time = 54.024s
2010-11-10 15:33:59.171: [    RACG][1] [618716][1][ora.whdb02.gsd]: GSD is not running on the local node
2010-11-10 15:33:59.171: [    RACG][1] [618716][1][ora.whdb02.gsd]: clsrcexecut: cmd = /oracle/app/oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 540 /oracle/app/oracle/product/10.2.0/crs/bin/gsdctl stat
2010-11-10 15:33:59.171: [    RACG][1] [618716][1][ora.whdb02.gsd]: clsrcexecut: rc = 1, time = 2.704s
2010-11-10 15:33:59.171: [    RACG][1] [618716][1][ora.whdb02.gsd]: end for resource = ora.whdb02.gsd, action = start, status = 1, time = 57.130s
2010-11-10 15:33:59.646: [    RACG][1] [618722][1][ora.whdb02.gsd]: clsrcgetprsrctx: prsr_init_ext returned rc = 3
2010-11-10 15:34:02.351: [    RACG][1] [618722][1][ora.whdb02.gsd]: GSD is not running on the local node
2010-11-10 15:40:29.176: [    RACG][1] [503954][1][ora.whdb02.gsd]: clsrcgetprsrctx: prsr_init_ext returned rc = 3
2010-11-10 15:41:20.184: [    RACG][1] [503954][1][ora.whdb02.gsd]: Failed to start GSD on local node
2010-11-10 15:41:20.184: [    RACG][1] [503954][1][ora.whdb02.gsd]: clsrcexecut: cmd = /oracle/app/oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 540 /oracle/app/oracle/product/10.2.0/crs/bin/gsdctl start
2010-11-10 15:41:20.184: [    RACG][1] [503954][1][ora.whdb02.gsd]: clsrcexecut: rc = 1, time = 51.007s
2010-11-10 15:41:22.888: [    RACG][1] [503954][1][ora.whdb02.gsd]: GSD is not running on the local node
2010-11-10 15:41:22.889: [    RACG][1] [503954][1][ora.whdb02.gsd]: clsrcexecut: cmd = /oracle/app/oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 540 /oracle/app/oracle/product/10.2.0/crs/bin/gsdctl stat
2010-11-10 15:41:22.889: [    RACG][1] [503954][1][ora.whdb02.gsd]: clsrcexecut: rc = 1, time = 2.703s
2010-11-10 15:41:22.889: [    RACG][1] [503954][1][ora.whdb02.gsd]: end for resource = ora.whdb02.gsd, action = start, status = 1, time = 54.106s
2010-11-10 15:41:23.373: [    RACG][1] [290992][1][ora.whdb02.gsd]: clsrcgetprsrctx: prsr_init_ext returned rc = 3
2010-11-10 15:41:26.078: [    RACG][1] [290992][1][ora.whdb02.gsd]: GSD is not running on the local node
2010-11-10 15:50:06.328: [    RACG][1] [442492][1][ora.whdb02.gsd]: clsrcgetprsrctx: prsr_init_ext returned rc = 3
2010-11-10 15:50:57.336: [    RACG][1] [442492][1][ora.whdb02.gsd]: Failed to start GSD on local node
2010-11-10 15:50:57.336: [    RACG][1] [442492][1][ora.whdb02.gsd]: clsrcexecut: cmd = /oracle/app/oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 540 /oracle/app/oracle/product/10.2.0/crs/bin/gsdctl start
2010-11-10 15:50:57.336: [    RACG][1] [442492][1][ora.whdb02.gsd]: clsrcexecut: rc = 1, time = 51.008s
2010-11-10 15:51:00.043: [    RACG][1] [442492][1][ora.whdb02.gsd]: GSD is not running on the local node
2010-11-10 15:51:00.043: [    RACG][1] [442492][1][ora.whdb02.gsd]: clsrcexecut: cmd = /oracle/app/oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 540 /oracle/app/oracle/product/10.2.0/crs/bin/gsdctl stat
2010-11-10 15:51:00.043: [    RACG][1] [442492][1][ora.whdb02.gsd]: clsrcexecut: rc = 1, time = 2.706s
2010-11-10 15:51:00.043: [    RACG][1] [442492][1][ora.whdb02.gsd]: end for resource = ora.whdb02.gsd, action = start, status = 1, time = 54.114s
2010-11-10 15:51:01.361: [    RACG][1] [618710][1][ora.whdb02.gsd]: clsrcgetprsrctx: prsr_init_ext returned rc = 3
2010-11-10 15:51:04.066: [    RACG][1] [618710][1][ora.whdb02.gsd]: GSD is not running on the local node

Error starting listener on one node in RAC:Error listening on....TNS-12545:

LSNRCTL> start LISTENER_CORPNG04
Starting /ora00/app/oracle/product/11/db1/bin/tnslsnr: please wait...
TNSLSNR for HPUX: Version 11.1.0.7.0 - Production
System parameter file is /ora00/app/oracle/product/11/db1/network/admin/listener.ora
Error listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=vir_corpng04)(PORT=1521)(IP=FIRST)))
TNS-12545: Connect failed because target host or object does not exist
TNS-12560: TNS:protocol adapter error
TNS-00515: Connect failed because target host or object does not exist
HPUX Error: 227: Can't assign requested address
Listener failed to start. See the error message(s) above...
LSNRCTL>
Plz help its production system
Thanks in Advance
Gagan

See link
Rgds

Scan-vip running only on one RAC node

Hi ,
While setting up RAC11.2 on Centos 5.7 , I was getting this error during the grid installation:
PRCR-1079 : Failed to start resource ora.scan1.vip
CRS-5005: IP Address: 192.168.100.208 is already in use in the network
CRS-2674: Start of 'ora.scan1.vip' on 'falcen6b' failed
CRS-2632: There are no more servers to try to place resource 'ora.scan1.vip' on that would satisfy its placement policy
PRCR-1079 : Failed to start resource ora.scan2.vip
CRS-5005: IP Address: 192.168.100.209 is already in use in the network
CRS-2674: Start of 'ora.scan2.vip' on 'falcen6b' failed
CRS-2632: There are no more servers to try to place resource 'ora.scan2.vip' on that would satisfy its placement policy
PRCR-1079 : Failed to start resource ora.scan3.vip
CRS-5005: IP Address: 192.168.100.210 is already in use in the network
CRS-2674: Start of 'ora.scan3.vip' on 'falcen6b' failed
CRS-2632: There are no more servers to try to place resource 'ora.scan3.vip' on that would satisfy its placement policy
I figured that the scan service is able to run only on one node at a time. When I stopped the service on rac1 and started it on rac2 the service is starting.
But I think for the grid installation the scan service has to simultaneously run on both the nodes.
How do I resolve it?
Any suggestions please.
PS - I am planning to try with the patch 11.0.2.3 but it will be a while till i get access to it.
Till then can someone suggest a workaround?

Hi Balazs Papp and onedbguru,
I was able to resolve that error by running the following command on rac2, now that part of the installer passed.
crsctl start res ora.scan1.vip
However the cluster verification utility is failing at the end of installer.
When I executed the below command, this is my output:
[oracle@falcen6a grid]$ ./runcluvfy.sh stage -post crsinst -n falcen6a,falcen6b -verbose
Performing post-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "falcen6a"
Destination Node Reachable?
falcen6a yes
falcen6b yes
Result: Node reachability check passed from node "falcen6a"
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Comment
falcen6b passed
falcen6a passed
Result: User equivalence check passed for user "oracle"
Checking time zone consistency...
Time zone consistency check passed.
Checking Cluster manager integrity...
Checking CSS daemon...
Node Name Status
falcen6b running
falcen6a running
Oracle Cluster Synchronization Services appear to be online.
Cluster manager integrity check passed
UDev attributes check for OCR locations started...
Result: UDev attributes check passed for OCR locations
UDev attributes check for Voting Disk locations started...
Result: UDev attributes check passed for Voting Disk locations
Check default user file creation mask
Node Name Available Required Comment
falcen6b 0022 0022 passed
falcen6a 0022 0022 passed
Result: Default user file creation mask check passed
Checking cluster integrity...
Cluster is divided into 2 partitions
Partition 1 consists of the following members:
Node Name
falcen6b
Partition 2 consists of the following members:
Node Name
falcen6a
Cluster integrity check failed. Cluster is divided into 2 partition(s).
Checking OCR integrity...
Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations
ERROR:
PRVF-4193 : Asm is not running on the following nodes. Proceeding with the remaining nodes.
Checking OCR config file "/etc/oracle/ocr.loc"...
OCR config file "/etc/oracle/ocr.loc" check successful
ERROR:
PRVF-4195 : Disk group for ocr location "+DATA" not available on the following nodes:
Checking size of the OCR location "+DATA" ...
Size check for OCR location "+DATA" successful...
OCR integrity check failed
Checking CRS integrity...
ERROR:
PRVF-5316 : Failed to retrieve version of CRS installed on node "falcen6b"
The Oracle clusterware is healthy on node "falcen6b"
The Oracle clusterware is healthy on node "falcen6a"
CRS integrity check failed
Checking node application existence...
Checking existence of VIP node application
Node Name Required Status Comment
falcen6b yes unknown failed
falcen6a yes unknown failed
Result: Check failed.
Checking existence of ONS node application
Node Name Required Status Comment
falcen6b no unknown ignored
falcen6a no online passed
Result: Check ignored.
Checking existence of GSD node application
Node Name Required Status Comment
falcen6b no unknown ignored
falcen6a no does not exist ignored
Result: Check ignored.
Checking existence of EONS node application
Node Name Required Status Comment
falcen6b no unknown ignored
falcen6a no online passed
Result: Check ignored.
Checking existence of NETWORK node application
Node Name Required Status Comment
falcen6b no unknown ignored
falcen6a no online passed
Result: Check ignored.
Checking Single Client Access Name (SCAN)...
SCAN VIP name Node Running? ListenerName Port Running?
falcen6-scan unknown false LISTENER 1521 false
WARNING:
PRVF-5056 : Scan Listener "LISTENER" not running
Checking name resolution setup for "falcen6-scan"...
SCAN Name IP Address Status Comment
falcen6-scan 192.168.100.210 passed
falcen6-scan 192.168.100.208 passed
falcen6-scan 192.168.100.209 passed
Verification of SCAN VIP and Listener setup failed
OCR detected on ASM. Running ACFS Integrity checks...
Starting check to see if ASM is running on all cluster nodes...
PRVF-5137 : Failure while checking ASM status on node "falcen6b"
Starting Disk Groups check to see if at least one Disk Group configured...
Disk Group Check passed. At least one Disk Group configured
Task ACFS Integrity check failed
Checking Oracle Cluster Voting Disk configuration...
Oracle Cluster Voting Disk configuration check passed
Checking to make sure user "oracle" is not in "root" group
Node Name Status Comment
falcen6b does not exist passed
falcen6a does not exist passed
Result: User "oracle" is not part of "root" group. Check passed
Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed
Checking if CTSS Resource is running on all nodes...
Check: CTSS Resource running on all nodes
Node Name Status
falcen6b passed
falcen6a passed
Result: CTSS resource check passed
Querying CTSS for time offset on all nodes...
Result: Query of CTSS for time offset passed
Check CTSS state started...
Check: CTSS state
Node Name State
falcen6b Observer
falcen6a Observer
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
The NTP configuration file "/etc/ntp.conf" is available on all nodes
NTP Configuration file check passed
Checking daemon liveness...
Check: Liveness for "ntpd"
Node Name Running?
falcen6b yes
falcen6a yes
Result: Liveness check passed for "ntpd"
Checking NTP daemon command line for slewing option "-x"
Check: NTP daemon command line
Node Name Slewing Option Set?
falcen6b yes
falcen6a yes
Result:
NTP daemon slewing option check passed
Checking NTP daemon's boot time configuration, in file "/etc/sysconfig/ntpd", for slewing option "-x"
Check: NTP daemon's boot time configuration
Node Name Slewing Option Set?
falcen6b yes
falcen6a yes
Result:
NTP daemon's boot time configuration check for slewing option passed
NTP common Time Server Check started...
NTP Time Server "133.243.236.19" is common to all nodes on which the NTP daemon is running
NTP Time Server "133.243.236.18" is common to all nodes on which the NTP daemon is running
NTP Time Server "210.173.160.86" is common to all nodes on which the NTP daemon is running
NTP Time Server ".LOCL." is common to all nodes on which the NTP daemon is running
Check of common NTP Time Server passed
Clock time offset check from NTP Time Server started...
Checking on nodes "[falcen6b, falcen6a]"...
Check: Clock time offset from NTP Time Server
Time Server: 133.243.236.19
Time Offset Limit: 1000.0 msecs
Node Name Time Offset Status
falcen6b 15.332 passed
falcen6a -1.503 passed
Time Server "133.243.236.19" has time offsets that are within permissible limits for nodes "[falcen6b, falcen6a]".
Time Server: 133.243.236.18
Time Offset Limit: 1000.0 msecs
Node Name Time Offset Status
falcen6b 15.115 passed
falcen6a -1.614 passed
Time Server "133.243.236.18" has time offsets that are within permissible limits for nodes "[falcen6b, falcen6a]".
Time Server: 210.173.160.86
Time Offset Limit: 1000.0 msecs
Node Name Time Offset Status
falcen6b 15.219 passed
falcen6a -1.527 passed
Time Server "210.173.160.86" has time offsets that are within permissible limits for nodes "[falcen6b, falcen6a]".
Time Server: .LOCL.
Time Offset Limit: 1000.0 msecs
Node Name Time Offset Status
falcen6b 0.0 passed
falcen6a 0.0 passed
Time Server ".LOCL." has time offsets that are within permissible limits for nodes "[falcen6b, falcen6a]".
Clock time offset check passed
Result: Clock synchronization check using Network Time Protocol(NTP) passed
Oracle Cluster Time Synchronization Services check passed
Post-check for cluster services setup was unsuccessful on all the nodes.
[oracle@falcen6a grid]$
Any suggestions?

ASM disk busy 99% only on one cluster node

Hello,
We have a three node Oracle RAC cluster. Our dba(s) called us and said they are getting OEM critical alers for an asm disk on one node only. I checked and the SAN attached drive does not show the same high utilization on either of the other two nodes. I checked the hardware and it seems fine. If the issue was with the SAN attached disk, we would be seeing the same errors on all three nodes since they share the same disks. The system crashed last week(alert dump in the +asm directories), and at the disk has been busy ever since. I asked if the dba reviewed the ADDM reports and he said he had and that there were no suspicious looking entries that would lead us to the root cause based on those reports. CPU utilization is fine. I am not sure where to look at this point and any help pointing me in the right direction would be appreciated. They do use RMAN, could there be a backup running using those disks only on one node? Has anyone ever seen this before?
Thank you,
Benita Ulisano
Unix/SAN Team
Chicago Public Schools
[email protected]

Hi Harish,
Thank you for responding. To answer your question, yes, the disks are all of the same spec and are shared among the three cluster node. The asm disk sdw1 is the one with the issue.
Problem Node: coefsdb02
three nodes in RAC cluster
coefsdb01, coefsdb02, coefsdb03
iostat results for all three nodes - same disk
coefsdb01
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sdw1 0.00 1.71 0.12 0.58 1.27 18.78 28.63 0.01 13.38 1.75 0.12
coefsdb02
sdw1 0.11 0.02 4.00 0.62 305.84 21.72 70.93 2.96 12.58 211.95 97.88
coefdb03
sdw1 0.21 0.01 4.70 0.33 224.05 13.52 47.22 0.05 10.11 6.15 3.09
The dba(s) run RMAN backups, but only on coefsdb01.
Benita

Configuring Scheduler only on single node in a clustered environment

Friends,
I have OIM 11gr2 environment in clustered mode with 6 nodes in it.
As per my client requirement, I need to configure scheduler only on one node and No on all other nodes. So, all schedule jobs OOTB and custom jobs should run only on one node.
What is the process for this?
Thanks,
MM

Check this: Managing the Scheduler - 11g Release 2 (11.1.2.1.0)
-Bikash

When one node reboot other node in RAC

Hi Friends,
I faced one situation where one node of RAC cluster had been rebooted by other node. This happen due to network interconnect link fluctuation.
Sep 13 16:23:48 kkvs1a su: [ID 810491 auth.crit] 'su admin' failed for wipro1 on /dev/pts/3
Sep 14 00:22:17 kkvs1a ixgbe: [ID 611667 kern.info] NOTICE: ixgbe3: link down
Sep 14 00:22:21 kkvs1a ixgbe: [ID 611667 kern.info] NOTICE: ixgbe3: link up, , full duplex
Sep 14 00:22:31 kkvs1a ixgbe: [ID 611667 kern.info] NOTICE: ixgbe1: link down
Sep 14 00:22:31 kkvs1a ixgbe: [ID 611667 kern.info] NOTICE: ixgbe3: link down
/opt/oracle/product/10.2.0/crs/log/node1/alertkk1a.log
==============================================
2013-09-14 00:22:05.180
[cssd(12561)]CRS-1612:node kk1b (2) at 50% heartbeat fatal, eviction in 14.251 seconds
2013-09-14 00:22:12.180
[cssd(12561)]CRS-1611:node kk1b (2) at 75% heartbeat fatal, eviction in 7.251 seconds
2013-09-14 00:22:13.180
[cssd(12561)]CRS-1611:node kk1b (2) at 75% heartbeat fatal, eviction in 6.251 seconds
2013-09-14 00:22:17.179
[cssd(12561)]CRS-1610:node kk1b (2) at 90% heartbeat fatal, eviction in 2.251 seconds
2013-09-14 00:22:18.180
[cssd(12561)]CRS-1610:node kkvs1b (2) at 90% heartbeat fatal, eviction in 1.251 seconds
This clearly shows CSSD of node kkvs1a has given node eviction message to kkvs1b node.
I got following messages on the instance which got rebooted:
ASM alert log:
Sat Sep 14 00:22:25 IST 2013
Error: KGXGN aborts the instance (6)
Sat Sep 14 00:22:25 IST 2013
Errors in file /opt/oracle/admin/+ASM/bdump/+asm2_lmon_8527.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
A network fluctuation shouldn't give reboot like this. Then why oracle design like this way? Is this a bug? My oracle version is: 10.2.0.5.0
Could you tell me the other possible situations when 1 RC instance reboots other RAC instacne.

What you are describing is the expected behaviour: if your interconnect fails, you will have a node eviction. Releases < 11.2.0.2 evict a node by reboot, which can fix the problem: the NIC may come up correctly when the machine re-starts. Releases >= 11.2.0.2 can often evict without a re-boot. But either way, if your interconnect goes down, a node must be evicted to prevent uncoordinated disc writes.
If you are interested, you can find some discussion and demos of this in a series of webcasts I've recorded,
Free Oracle Database Tutorials for Administration and Developers
If you really don't like this behaviour and the problems are transient, you can try 'raising the CSS MISSCOUNT parameter.
John Watson
Oracle Certified Master DBA

Direct Traffic to one node in Cloud Service

Is there a way to make a cloud service (web role) always use one of the nodes, there are two in this scenario, and only direct to the other one if the first one gets shut down. We have a piece of legacy software we use in our Cloud Service and we are waiting
on it being updated so it can work across the load balancer, but currently it does not. So we only have one node in the service. I would like to have two and see if we can always hit node 1 for example, and then hit node 2 if 1 is down. I am assuming, maybe
wrongly, that this would still be within SLA, and if maintenance was happening then both nodes would not get shutdown. Is it possible to maybe have a web role class which controls the load balancer traffic?
Thanks
Eamonn

Hi Eamonn,
Thanks for your posting!
>>Is there a way to make a cloud service (web role) always use one of the nodes, there are two in this scenario,...
Base on my experience, if we have one or more instances of a VM (Web or Worker role), the traffic will be distributed amongst the instances. We isn't allowed to specify which instances. Please see David's psot(http://stackoverflow.com/a/12726613
>> I would like to have two and see if we can always hit node 1 for example, and then hit node 2 if 1 is down. I am assuming, maybe.....
About this requirement, I suggest you could try to those steps:
1.create a web role with adding a web service or create a web role using WCF service.
2.create a service with the instance information, like this:
[WebMethod]
public ReturnResult ReverseString(string value)
ReturnResult rr = new ReturnResult();
rr.ReturnString = new string(value);
rr.HostName = RoleEnvironment.CurrentRoleInstance.Id;;
return rr;
//Class
public class ReturnResult
private String returnString;
public String ReturnString
get { return returnString; }
set { returnString = value; }
private String hostName;
public String HostName
get { return hostName; }
set { hostName = value; }
3.Host this service on azure cloud service
4.Create a new project whatever it is webpage or console application.
5.using the Thread Pool or BackgroundWorker to send the concurrent request.
6.Check the results list and host name.
After made it, you could get the every requests hot which instances .
If you'd like to custom the Azure loadbalancer, I recommend you could refer to those documents:
http://msdn.microsoft.com/en-us/library/azure/jj151530.aspx
http://blogs.msdn.com/b/kdot/archive/2013/06/29/implementing-custom-load-balancing-with-sticky-connections-on-windows-azure.aspx
Regards,
Will
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey.

Database starting on one node.

Hi All,
After reboot of server and ssh key gen. My RAC instance is running only on one node.
What can be problem?
Thanks.

Only you know what can be problem, because only you have valuable inforamation. :)
- Have you checked CRS-related logs?
- Have you checked Oracle alert log and related background process trace files?
If anything interesting found, post and let us know.

Ora.reco.acfsvol.acfs only on one node on RAC on ODA

Similar Messages

Maybe you are looking for