Failure 1 contacting cssd daemon

HI
we have solaris server10 - 2 node RAC db with 10.2.0.4 version .
When i check crsctl check crs , the output is showing as
crsctl check crs
Failure 1 contacting CSS daemon
CRS appears healthy
EVM appears healthy
when i check crs_stat -t -v , the output is as follows ,
crs_stat -t -v
Name Type R/RA F/FT Target State Host
ora....SM1.asm application 0/5 0/0 ONLINE ONLINE server1
ora....R1.lsnr application 0/5 0/0 ONLINE ONLINE server1
ora....er1.vip application 0/0 0/0 ONLINE ONLINE server1
ora....SM2.asm application 0/5 0/0 ONLINE ONLINE server2
ora....R2.lsnr application 0/5 0/0 ONLINE ONLINE server2
ora....er2.gsd application 0/5 0/0 ONLINE ONLINE server2
ora....er2.ons application 0/3 0/0 ONLINE ONLINE server2
ora....er2.vip application 0/0 0/0 ONLINE ONLINE server2
ora....ogdb.db application 0/1 0/1 ONLINE ONLINE server1
ora....ebdb.cs application 0/0 0/1 ONLINE ONLINE server1
ora....db1.srv application 0/0 0/0 ONLINE ONLINE server1
ora....db2.srv application 0/0 0/0 ONLINE ONLINE server2
ora....b1.inst application 0/5 0/0 ONLINE ONLINE server1
ora....b2.inst application 0/5 0/0 ONLINE ONLINE server2
when ichecked the ocssd.log file , it show like the following
CSSD]2011-09-16 07:44:50.842 [13] >TRACE: clscsendx: (1008d4300) Connection not active
[    CSSD]2011-09-16 07:44:50.842 [13] >TRACE: clssgmSendClient: Send failed rc 6, con (1008d4300), client (1008d4880),
proc (0)
[    CSSD]2011-09-16 07:45:10.688 [13] >TRACE: clscsendx: (1008afc00) Connection not active
[    CSSD]2011-09-16 07:45:10.688 [13] >TRACE: clssgmSendClient: Send failed rc 6, con (1008afc00), client (1008afeb0),
proc (0)
[    CSSD]2011-09-16 07:45:10.689 [13] >TRACE: clscsendx: (1008b0170) Connection not active
i am not understanding where is the fault, and where to check and diagnose the problem.
Reg
kumar

Hi
Later i checked alert log and found the following errors
ORA-01114: IO error writing block to file 5 (block # 281197)
ORA-29701: unable to connect to Cluster Manager
ORA-01114: IO error writing block to file 5 (block # 281197)
ORA-29701: unable to connect to Cluster Manager
ORA-01114: IO error writing block to file 5 (block # 281197)
ORA-29701: unable to connect to Cluster Manager
What could be the issue with clusterware.
Reg
kumar

Similar Messages

Primus: fatal: failure contacting bumblebee daemon [SOLVED]

I just did a pacman -Syu which broke my system as usual. This time I get the above error message when trying to use primusrun:
primus: fatal: failure contacting bumblebee daemon
Trying to run glxshperes gives a slightly different message:
$ primusrun glxspheres
primus: fatal: Bumblebee daemon reported: error: Could not load GPU driver
and via optirun, just for the heck of it:
$ optirun glxgears
[ 2702.399865] [ERROR]Cannot access secondary GPU - error: Could not load GPU driver
[ 2702.399894] [ERROR]Aborting because fallback start is disabled.
<<SOLUTION: Replace all the *-bumblebee packages by * packages, this means enabling multiverse repo and drawing all of it from official repos instead of anything from AUR. Examples: nvidia instead of nvidia-bumblebee, primus instead of primus-git, nvidia-utils instead of nvidia-utils-bumblebee, lib32-nvidia-utils instead of lib32-nvidia-utils-bumblebee. Some time ago the 'official' instructions were exactly the other way round, but it seems this was now completely reverted.>>
I noticed:
1) primus in official repos (I was using git so far) causes a timeout from about 40 mirrors until it finds one where the download works. I hope that's no bad sign of some sort?
2) bumblebeed update created new bumblebee.conf.pacnew and xorg.conf.nvidia.pacnew
3) off-topic: When I clicked 'logout' in XFCE's applications menu, usually I get a box where I can pick whether I wanna logout/restart/shutdown etc, but this time it just straightly logged me out (makes sense I guess).
4) after rebooting my system I saw I think two systemd error messages pop up for a split second, that usually did not occur.
5) Xorg.8.log had the line "[633217.731] (WW) NVIDIA: This server has an unsupported input driver ABI version (have 19.1, need < 19.0). The driver will continue to load, but may behave strangely.", is this responsible maybe? Version conflict between X and nvidia?
I tried:
I restarted after the pacman -Syu update.
I tried primusrun-git from AUR and primusrun from <community> but both exit with the error message.
I tried the new conf files (copied the pacnew files to the normal filenames) but they don't work either.
I checked:
Bumbleed is running, systemctl shows
bumblebeed.service loaded active running Bumblebee C Daemon
and lspci | grep VGA shows the usual
00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor Graphics Controller (rev 09)
01:00.0 VGA compatible controller: NVIDIA Corporation Device 0fd4 (rev a1)
(After I try to start something with primusrun, the indicator light switches on and remains on, I can turn it off again with the usual
echo OFF > /proc/acpi/bbswitch
dmesg has these:
[ 94.057043] nvidia: disagrees about version of symbol pv_mmu_ops
[ 94.057047] nvidia: Unknown symbol pv_mmu_ops (err -22)
echoing 'OFF' to bbswitch seems fine though (it worked, as I mentioned)
[ 16.326834] bbswitch: version 0.6
[ 16.326840] bbswitch: Found integrated VGA device 0000:00:02.0: \_SB_.PCI0.GFX0
[ 16.326843] bbswitch: Found discrete VGA device 0000:01:00.0: \_SB_.PCI0.PEG0.PEGP
[ 16.326903] bbswitch: detected an Optimus _DSM function
[ 16.326908] bbswitch: Succesfully loaded. Discrete card 0000:01:00.0 is on
[ 16.327995] bbswitch: disabling discrete graphics
Here is /var/log/Xorg.8.log (trimmed):
X.Org X Server 1.14.1
[633217.228] X Protocol Version 11, Revision 0
[633217.228] Build Operating System: Linux 3.8.7-1-ARCH x86_64
[633217.229] (++) Using config file: "/etc/bumblebee/xorg.conf.nvidia"
[633217.229] (==) Using config directory: "/etc/X11/xorg.conf.d"
[633217.293] (++) ModulePath set to "/usr/lib/nvidia-bumblebee/xorg/,/usr/lib/xorg/modules"
[633217.295] Initializing built-in extension XVideo
[633217.295] Initializing built-in extension XVideo-MotionCompensation
[633217.295] Initializing built-in extension XFree86-VidModeExtension
[633217.295] Initializing built-in extension XFree86-DGA
[633217.295] Initializing built-in extension XFree86-DRI
[633217.295] Initializing built-in extension DRI2
[633217.295] (II) LoadModule: "glx"
[633217.308] (II) Loading /usr/lib/nvidia-bumblebee/xorg/modules/extensions/libglx.so
[633217.680] (II) Module glx: vendor="NVIDIA Corporation"
[633217.680] compiled for 4.0.2, module version = 1.0.0
[633217.681] Module class: X.Org Server Extension
[633217.681] (II) NVIDIA GLX Module 313.26 Wed Feb 27 13:10:40 PST 2013
[633217.681] Loading extension GLX
[633217.681] (II) LoadModule: "nvidia"
[633217.705] (II) Loading /usr/lib/xorg/modules/drivers/nvidia_drv.so
[633217.731] (II) Module nvidia: vendor="NVIDIA Corporation"
[633217.731] compiled for 4.0.2, module version = 1.0.0
[633217.731] Module class: X.Org Video Driver
[633217.731] (WW) NVIDIA: This server has an unsupported input driver ABI version (have 19.1, need < 19.0). The driver will continue to load, but may behave strangely.
[633217.732] (II) NVIDIA dlloader X Driver 313.26 Wed Feb 27 12:52:26 PST 2013
[633217.732] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[633217.732] (--) using VT number 7
[633217.733] (II) NVIDIA(0): Creating default Display subsection in Screen section
"Default Screen Section" for depth/fbbpp 24/32
[633217.733] (==) NVIDIA(0): Depth 24, (==) framebuffer bpp 32
[633217.733] (==) NVIDIA(0): RGB weight 888
[633217.733] (==) NVIDIA(0): Default visual is TrueColor
[633217.733] (==) NVIDIA(0): Using gamma correction (1.0, 1.0, 1.0)
[633217.733] (**) NVIDIA(0): Option "NoLogo" "true"
[633217.733] (**) NVIDIA(0): Option "UseEDID" "false"
[633217.733] (**) NVIDIA(0): Option "ConnectedMonitor" "CRT-0"
[633217.734] (**) NVIDIA(0): Enabling 2D acceleration
[633217.734] (**) NVIDIA(0): ConnectedMonitor string: "CRT-0"
[633217.734] (**) NVIDIA(0): Ignoring EDIDs
[633218.796] (II) NVIDIA(0): Implicitly enabling NoScanout
[633218.796] (WW) NVIDIA(0): Failed to enable display hotplug notification
[633218.799] (II) NVIDIA(0): NVIDIA GPU GeForce GTX 660M (GK107) at PCI:1:0:0 (GPU-0)
[633218.799] (--) NVIDIA(0): Memory: 2097152 kBytes
[633218.799] (--) NVIDIA(0): VideoBIOS: 80.07.27.00.05
[633218.799] (II) NVIDIA(0): Detected PCI Express Link width: 16X
[633218.799] (--) NVIDIA(0): Valid display device(s) on GeForce GTX 660M at PCI:1:0:0
[633218.799] (--) NVIDIA(0): none
[633218.799] (II) NVIDIA(0): Validated MetaModes:
[633218.799] (II) NVIDIA(0): "nvidia-auto-select"
[633218.799] (II) NVIDIA(0): Virtual screen size determined to be 640 x 480
[633218.799] (WW) NVIDIA(0): Unable to get display device for DPI computation.
[633218.799] (==) NVIDIA(0): DPI set to (75, 75); computed from built-in default
[633218.799] (--) Depth 24 pixmap format is 32 bpp
[633218.799] (II) NVIDIA: Using 3072.00 MB of virtual memory for indirect memory
[633218.799] (II) NVIDIA: access.
[633218.805] (II) NVIDIA(0): ACPI: failed to connect to the ACPI event daemon; the daemon
[633218.805] (II) NVIDIA(0): may not be running or the "AcpidSocketPath" X
[633218.805] (II) NVIDIA(0): configuration option may not be set correctly. When the
[633218.805] (II) NVIDIA(0): ACPI event daemon is available, the NVIDIA X driver will
[633218.805] (II) NVIDIA(0): try to use it to receive ACPI event notifications. For
[633218.805] (II) NVIDIA(0): details, please see the "ConnectToAcpid" and
[633218.805] (II) NVIDIA(0): "AcpidSocketPath" X configuration options in Appendix B: X
[633218.805] (II) NVIDIA(0): Config Options in the README.
[633218.806] (II) NVIDIA(0): Setting mode "nvidia-auto-select"
[633218.857] Loading extension NV-GLX
[633218.865] (==) NVIDIA(0): Disabling shared memory pixmaps
[633218.865] (==) NVIDIA(0): Backing store disabled
[633218.865] (==) NVIDIA(0): Silken mouse enabled
[633218.866] (==) NVIDIA(0): DPMS enabled
[633218.866] Loading extension NV-CONTROL
[633218.866] (II) Loading sub module "dri2"
[633218.866] (II) LoadModule: "dri2"
[633218.866] (II) Module "dri2" already built-in
[633218.866] (II) NVIDIA(0): [DRI2] Setup complete
[633218.866] (II) NVIDIA(0): [DRI2] VDPAU driver: nvidia
[633218.866] (--) RandR disabled
[633218.879] (II) Initializing extension GLX
Any ideas?
PS: In this light I was wondering whether the https://wiki.archlinux.org/index.php/Bumblebee article is still up to date or might need reworking of some sort (I don't have any actual ideas to base this on though, was just a thought)
PPS: https://bbs.archlinux.org/viewtopic.php?pid=1264430
Last edited by Jindur (2013-04-27 18:10:21)

iambig wrote:
Hi,
i haved some trouble too, from what i understood
nvidia-bumblebee
is deprecated, you should install
nvidia
otherwise i am not still able to use optirun/primus.
Hm, so far it was emphasized NOT to install nvidia, because that would actually break it, but to use nvidia-bumblebee instead.
Do you maybe have a link to information saying that it is now exactly the other way round?
Has 'nvidia' been confirmed to work with bumblebee by anyone? Does it work for you?
Edit: I just saw in my Xorg.8.log:
[633217.731] (WW) NVIDIA: This server has an unsupported input driver ABI version (have 19.1, need < 19.0). The driver will continue to load, but may behave strangely.
Maybe this has to do with it? It seems there might be a version conflict between X and nvidia drivers?
If anyone can confirm that the basic 'nvidia' packages are indeed replacing 'nvidia-bumblebee' now, instead of the other way round, I'll give them a try
Last edited by Jindur (2013-04-26 12:30:37)

Failure 1 contacting CSS daemon

hi all
please provide me the solution
when iam checking crs status i got error
-a $crsctl check crs
Failure 1 contacting CSS daemon
CRS appears healthy
EVM appears healthy
As iam new to rac
please give me step by step solution

-a $crsctl check crs
Failure 1 contacting CSS daemon
CRS appears healthy
EVM appears healthy
-a $ps -ef|grep crs
oracle 2368 2367 0 Jun 27 ? 142:28 /u01/crs/bin/oclsomon.bin
root 2189 1766 0 Jun 27 ? 1053:51 /u01/crs/bin/crsd.bin reboot
oracle 3404 1 0 Jun 27 ? 0:00 /u01/crs/opmn/bin/ons -d
root 1766 1 0 Jun 27 ? 0:00 /bin/sh /etc/init.d/init.crsd
run
oracle 2138 1764 0 Jun 27 ? 0:00 sh -c sh -c 'ulimit -c unlimi
ted; cd /u01/crs/log/sun4/evmd; exec /u01/crs/bin/
oracle 2367 2366 0 Jun 27 ? 0:00 /bin/sh -c cd /u01/crs/log/su
n4/cssd/oclsomon; ulimit -c unlimited; /u01/crs/bi
oracle 2366 2216 0 Jun 27 ? 0:00 sh -c /bin/sh -c 'cd /u01/crs
/log/sun4/cssd/oclsomon; ulimit -c unlimited; /u01
oracle 2509 2143 0 Jun 27 ? 2:49 /u01/crs/bin/evmlogger.bin -o
/u01/crs/evm/log/evmlogger.info -l /u01/crs/evm/l
oracle 2143 2138 0 Jun 27 ? 64:18 /u01/crs/bin/evmd.bin
oracle 3406 3404 0 Jun 27 ? 6:13 /u01/crs/opmn/bin/ons -d
root 2357 2200 0 Jun 27 ? 24:56 /u01/crs/bin/oprocd.bin run -
t 1000 -m 500 -f
oracle 2432 2243 0 Jun 27 ? 921:19 /u01/crs/bin/ocssd.bin
oracle 23823 23572 0 11:55:32 pts/1 0:00 grep crs
-a $

SIM Failure contact your service provider?

I have a newly purchased Dell E6420 with the DW5800 4G (LTE-3G) Mobilebroadband card embedded. I downloaded and installed the VZ Access Manager Version 7.6.6.5 which applies to the installed card.
While activating I am getting a SIM Failure contact your service provider. Help....

Hello A1AJake! I can help get your embedded card activated. It's best to start by reviewing your account. I'll need you to send me a personal message through the VZW forum. I'll need your full name and mobile number in order to assist. Thanks!

[Bumblebee]Failed to initialize the NVIDIA GPU at PCI:1:0:0.

Hi!
I've instaled Arch about a month ago and now wanted to check my nvidia card using Bumblebee, but have some errors…
[wicu@arch:~]$ optirun -vv minecraft
[ 4859.259278] [DEBUG]Reading file: /etc/bumblebee/bumblebee.conf
[ 4859.404191] [DEBUG]optirun version 3.0.1 starting...
[ 4859.404220] [DEBUG]Active configuration:
[ 4859.404226] [DEBUG] bumblebeed config file: /etc/bumblebee/bumblebee.conf
[ 4859.404242] [DEBUG] X display: :8
[ 4859.404254] [DEBUG] LD_LIBRARY_PATH: /usr/lib/nvidia-bumblebee:/usr/lib32/nvidia-bumblebee
[ 4859.404265] [DEBUG] Socket path: /var/run/bumblebee.socket
[ 4859.404276] [DEBUG] VGL Compression: proxy
[ 4859.536971] [INFO]Response: No - error: [XORG] (EE) NVIDIA(0): Failed to initialize the NVIDIA GPU at PCI:1:0:0. Please
[ 4859.537027] [ERROR]Cannot access secondary GPU - error: [XORG] (EE) NVIDIA(0): Failed to initialize the NVIDIA GPU at PCI:1:0:0. Please
[ 4859.537053] [DEBUG]Socket closed.
[ 4859.537086] [ERROR]Aborting because fallback start is disabled.
[ 4859.537094] [DEBUG]Killing all remaining processes.
[wicu@arch:~]$ sudo journalctl // here is only part when I tried use bumblebee
Nov 26 17:48:43 arch kernel: NVRM: failed to copy vbios to system memory.
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238670] [WARN][XORG] (WW) `fonts.dir' not found (or not valid) in "/usr/share/fonts/100dpi/".
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238729] [WARN][XORG] (WW) `fonts.dir' not found (or not valid) in "/usr/share/fonts/75dpi/".
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238765] [WARN][XORG] (WW) Open ACPI failed (/var/run/acpid.socket) (No such file or directory)
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238819] [ERROR][XORG] (EE) NVIDIA(0): Failed to initialize the NVIDIA GPU at PCI:1:0:0. Please
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238826] [ERROR][XORG] (EE) NVIDIA(0): check your system's kernel log for additional error
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238830] [ERROR][XORG] (EE) NVIDIA(0): messages and refer to Chapter 8: Common Problems in the
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238835] [ERROR][XORG] (EE) NVIDIA(0): README for additional information.
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238841] [ERROR][XORG] (EE) NVIDIA(0): Failed to initialize the NVIDIA graphics device!
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238847] [ERROR][XORG] (EE) NVIDIA(0): Failing initialization of X screen 0
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238853] [ERROR][XORG] (EE) Screen(s) found, but none have a usable configuration.
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238858] [ERROR][XORG] (EE)
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238863] [ERROR][XORG] (EE) Please also check the log file at "/var/log/Xorg.8.log" for additional information.
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238868] [ERROR][XORG] (EE)
Nov 26 17:48:43 arch bumblebeed[303]: [ 4139.238873] [ERROR]X did not start properly
Nov 26 17:48:43 arch kernel: NVRM: RmInitAdapter failed! (0x30:0xffffffff:739)
Nov 26 17:48:43 arch kernel: NVRM: rm_init_adapter(0) failed
// xorg log
[ 4859.510]
X.Org X Server 1.13.0
Release Date: 2012-09-05
[ 4859.511] X Protocol Version 11, Revision 0
[ 4859.511] Build Operating System: Linux 3.6.3-1-ARCH x86_64
[ 4859.511] Current Operating System: Linux arch 3.6.7-1-ARCH #1 SMP PREEMPT Sun Nov 18 10:11:22 CET 2012 x86_64
[ 4859.511] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-linux root=UUID=70a6cff9-bcf7-42eb-9ba1-73953e023617 ro quiet
[ 4859.511] Build Date: 08 November 2012 07:09:29PM
[ 4859.511]
[ 4859.511] Current version of pixman: 0.28.0
[ 4859.511] Before reporting problems, check http://wiki.x.org
to make sure that you have the latest version.
[ 4859.511] Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[ 4859.511] (==) Log file: "/var/log/Xorg.8.log", Time: Mon Nov 26 18:00:43 2012
[ 4859.511] (++) Using config file: "/etc/bumblebee/xorg.conf.nvidia"
[ 4859.511] (==) Using config directory: "/etc/X11/xorg.conf.d"
[ 4859.511] (==) ServerLayout "Layout0"
[ 4859.512] (==) No screen section available. Using defaults.
[ 4859.512] (**) |-->Screen "Default Screen Section" (0)
[ 4859.512] (**) | |-->Monitor "<default monitor>"
[ 4859.512] (==) No device specified for screen "Default Screen Section".
Using the first device section listed.
[ 4859.512] (**) | |-->Device "Device1"
[ 4859.512] (==) No monitor specified for screen "Default Screen Section".
Using a default monitor configuration.
[ 4859.512] (**) Option "AutoAddDevices" "false"
[ 4859.512] (**) Not automatically adding devices
[ 4859.512] (==) Automatically enabling devices
[ 4859.512] (==) Automatically adding GPU devices
[ 4859.512] (WW) `fonts.dir' not found (or not valid) in "/usr/share/fonts/100dpi/".
[ 4859.512] Entry deleted from font path.
[ 4859.512] (Run 'mkfontdir' on "/usr/share/fonts/100dpi/").
[ 4859.512] (WW) `fonts.dir' not found (or not valid) in "/usr/share/fonts/75dpi/".
[ 4859.512] Entry deleted from font path.
[ 4859.512] (Run 'mkfontdir' on "/usr/share/fonts/75dpi/").
[ 4859.512] (==) FontPath set to:
/usr/share/fonts/misc/,
/usr/share/fonts/TTF/,
/usr/share/fonts/OTF/,
/usr/share/fonts/Type1/
[ 4859.512] (++) ModulePath set to "/usr/lib/nvidia-bumblebee/xorg/,/usr/lib/xorg/modules"
[ 4859.512] (==) |-->Input Device "<default pointer>"
[ 4859.512] (==) |-->Input Device "<default keyboard>"
[ 4859.512] (==) The core pointer device wasn't specified explicitly in the layout.
Using the default mouse configuration.
[ 4859.512] (==) The core keyboard device wasn't specified explicitly in the layout.
Using the default keyboard configuration.
[ 4859.512] (II) Loader magic: 0x7fcc40
[ 4859.512] (II) Module ABI versions:
[ 4859.512] X.Org ANSI C Emulation: 0.4
[ 4859.513] X.Org Video Driver: 13.1
[ 4859.513] X.Org XInput driver : 18.0
[ 4859.513] X.Org Server Extension : 7.0
[ 4859.513] (II) config/udev: Adding drm device (/dev/dri/card0)
[ 4859.513] setversion 1.4 failed
[ 4859.516] (--) PCI:*(0:1:0:0) 10de:0de9:17aa:3901 rev 161, Mem @ 0xd2000000/16777216, 0xc0000000/268435456, 0xd0000000/33554432, I/O @ 0x00003000/128, BIOS @ 0x????????/524288
[ 4859.516] (WW) Open ACPI failed (/var/run/acpid.socket) (No such file or directory)
[ 4859.516] Initializing built-in extension Generic Event Extension
[ 4859.516] Initializing built-in extension SHAPE
[ 4859.516] Initializing built-in extension MIT-SHM
[ 4859.516] Initializing built-in extension XInputExtension
[ 4859.516] Initializing built-in extension XTEST
[ 4859.516] Initializing built-in extension BIG-REQUESTS
[ 4859.516] Initializing built-in extension SYNC
[ 4859.516] Initializing built-in extension XKEYBOARD
[ 4859.516] Initializing built-in extension XC-MISC
[ 4859.516] Initializing built-in extension SECURITY
[ 4859.516] Initializing built-in extension XINERAMA
[ 4859.516] Initializing built-in extension XFIXES
[ 4859.516] Initializing built-in extension RENDER
[ 4859.516] Initializing built-in extension RANDR
[ 4859.516] Initializing built-in extension COMPOSITE
[ 4859.516] Initializing built-in extension DAMAGE
[ 4859.516] Initializing built-in extension MIT-SCREEN-SAVER
[ 4859.516] Initializing built-in extension DOUBLE-BUFFER
[ 4859.516] Initializing built-in extension RECORD
[ 4859.516] Initializing built-in extension DPMS
[ 4859.516] Initializing built-in extension X-Resource
[ 4859.516] Initializing built-in extension XVideo
[ 4859.516] Initializing built-in extension XVideo-MotionCompensation
[ 4859.516] Initializing built-in extension XFree86-VidModeExtension
[ 4859.516] Initializing built-in extension XFree86-DGA
[ 4859.516] Initializing built-in extension XFree86-DRI
[ 4859.516] Initializing built-in extension DRI2
[ 4859.516] (II) LoadModule: "glx"
[ 4859.516] (II) Loading /usr/lib/nvidia-bumblebee/xorg/modules/extensions/libglx.so
[ 4859.526] (II) Module glx: vendor="NVIDIA Corporation"
[ 4859.526] compiled for 4.0.2, module version = 1.0.0
[ 4859.526] Module class: X.Org Server Extension
[ 4859.526] (II) NVIDIA GLX Module 310.19 Thu Nov 8 01:12:43 PST 2012
[ 4859.526] Loading extension GLX
[ 4859.526] (II) LoadModule: "nvidia"
[ 4859.526] (II) Loading /usr/lib/xorg/modules/drivers/nvidia_drv.so
[ 4859.526] (II) Module nvidia: vendor="NVIDIA Corporation"
[ 4859.526] compiled for 4.0.2, module version = 1.0.0
[ 4859.526] Module class: X.Org Video Driver
[ 4859.526] (II) LoadModule: "mouse"
[ 4859.527] (II) Loading /usr/lib/xorg/modules/input/mouse_drv.so
[ 4859.527] (II) Module mouse: vendor="X.Org Foundation"
[ 4859.527] compiled for 1.13.0, module version = 1.8.1
[ 4859.527] Module class: X.Org XInput Driver
[ 4859.527] ABI class: X.Org XInput driver, version 18.0
[ 4859.527] (II) LoadModule: "kbd"
[ 4859.527] (WW) Warning, couldn't open module kbd
[ 4859.527] (II) UnloadModule: "kbd"
[ 4859.527] (II) Unloading kbd
[ 4859.527] (EE) Failed to load module "kbd" (module does not exist, 0)
[ 4859.527] (II) NVIDIA dlloader X Driver 310.19 Thu Nov 8 00:53:33 PST 2012
[ 4859.527] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[ 4859.527] (--) using VT number 7
[ 4859.527] (II) Loading sub module "wfb"
[ 4859.527] (II) LoadModule: "wfb"
[ 4859.527] (II) Loading /usr/lib/xorg/modules/libwfb.so
[ 4859.527] (II) Module wfb: vendor="X.Org Foundation"
[ 4859.527] compiled for 1.13.0, module version = 1.0.0
[ 4859.527] ABI class: X.Org ANSI C Emulation, version 0.4
[ 4859.527] (II) Loading sub module "ramdac"
[ 4859.527] (II) LoadModule: "ramdac"
[ 4859.527] (II) Module "ramdac" already built-in
[ 4859.527] (II) NVIDIA(0): Creating default Display subsection in Screen section
"Default Screen Section" for depth/fbbpp 24/32
[ 4859.527] (==) NVIDIA(0): Depth 24, (==) framebuffer bpp 32
[ 4859.527] (==) NVIDIA(0): RGB weight 888
[ 4859.527] (==) NVIDIA(0): Default visual is TrueColor
[ 4859.527] (==) NVIDIA(0): Using gamma correction (1.0, 1.0, 1.0)
[ 4859.527] (**) NVIDIA(0): Option "NoLogo" "true"
[ 4859.528] (**) NVIDIA(0): Option "UseEDID" "false"
[ 4859.528] (**) NVIDIA(0): Option "ConnectedMonitor" "DFP"
[ 4859.528] (**) NVIDIA(0): Enabling 2D acceleration
[ 4859.528] (**) NVIDIA(0): ConnectedMonitor string: "DFP"
[ 4859.528] (**) NVIDIA(0): Ignoring EDIDs
[ 4859.534] (EE) NVIDIA(0): Failed to initialize the NVIDIA GPU at PCI:1:0:0. Please
[ 4859.534] (EE) NVIDIA(0): check your system's kernel log for additional error
[ 4859.534] (EE) NVIDIA(0): messages and refer to Chapter 8: Common Problems in the
[ 4859.534] (EE) NVIDIA(0): README for additional information.
[ 4859.534] (EE) NVIDIA(0): Failed to initialize the NVIDIA graphics device!
[ 4859.534] (EE) NVIDIA(0): Failing initialization of X screen 0
[ 4859.534] (II) UnloadModule: "nvidia"
[ 4859.534] (II) UnloadSubModule: "wfb"
[ 4859.534] (EE) Screen(s) found, but none have a usable configuration.
[ 4859.534]
Fatal server error:
[ 4859.534] no screens found
[ 4859.534] (EE)
Please consult the The X.Org Foundation support
at http://wiki.x.org
for help.
[ 4859.534] (EE) Please also check the log file at "/var/log/Xorg.8.log" for additional information.
[ 4859.534] (EE)
[ 4859.534] Server terminated with error (1). Closing log file.
[wicu@arch:~]$ lspci | grep VGA
3:00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
14:01:00.0 VGA compatible controller: NVIDIA Corporation GF108 [GeForce GT 630M] (rev a1)
[wicu@arch:~]$ primusrun minecraft
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 9: reading configurations from ~/.fonts.conf is deprecated.
asdf
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 9: reading configurations from ~/.fonts.conf is deprecated.
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 9: reading configurations from ~/.fonts.conf is deprecated.
27 achievements
208 recipes
Setting user: Druedain, -4246576297828159265
primus: fatal: failure contacting bumblebee daemon
[18:05:39 - wicu@arch:~]$ primusrun minecraft
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 9: reading configurations from ~/.fonts.conf is deprecated.
asdf
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 9: reading configurations from ~/.fonts.conf is deprecated.
27 achievements
208 recipes
Setting user: Druedain, -7896572405277476304
primus: fatal: failure contacting bumblebee daemon
[wicu@arch:~]$ systemctl | grep bumblebee
35:bumblebeed.service

[wicu@arch:~]$ dmesg | grep nvidia
782:[ 6.776816] nvidia: module license 'NVIDIA' taints kernel.
784:[ 6.784411] nvidia 0000:01:00.0: enabling device (0006 -> 0007)
[wicu@arch:~]$ cat /etc/bumblebee/bumblebee.conf
# Configuration file for Bumblebee. Values should **not** be put between quotes
## Server options. Any change made in this section will need a server restart
# to take effect.
[bumblebeed]
# The secondary Xorg server DISPLAY number
VirtualDisplay=:8
# Should the unused Xorg server be kept running? Set this to true if waiting
# for X to be ready is too long and don't need power management at all.
KeepUnusedXServer=false
# The name of the Bumbleblee server group name (GID name)
ServerGroup=bumblebee
# Card power state at exit. Set to false if the card shoud be ON when Bumblebee
# server exits.
TurnCardOffAtExit=false
# The default behavior of '-f' option on optirun. If set to "true", '-f' will
# be ignored.
NoEcoModeOverride=false
# The Driver used by Bumblebee server. If this value is not set (or empty),
# auto-detection is performed. The available drivers are nvidia and nouveau
# (See also the driver-specific sections below)
Driver=
## Client options. Will take effect on the next optirun executed.
[optirun]
# The method used for VirtualGL to transport frames between X servers.
# Possible values are proxy, jpeg, rgb, xv and yuv.
VGLTransport=proxy
# Should the program run under optirun even if Bumblebee server or nvidia card
# is not available?
AllowFallbackToIGC=false
# Driver-specific settings are grouped under [driver-NAME]. The sections are
# parsed if the Driver setting in [bumblebeed] is set to NAME (or if auto-
# detection resolves to NAME).
# PMMethod: method to use for saving power by disabling the nvidia card, valid
# values are: auto - automatically detect which PM method to use
# bbswitch - new in BB 3, recommended if available
# switcheroo - vga_switcheroo method, use at your own risk
# none - disable PM completely
# https://github.com/Bumblebee-Project/Bumblebee/wiki/Comparison-of-PM-methods
## Section with nvidia driver specific options, only parsed if Driver=nvidia
[driver-nvidia]
# Module name to load, defaults to Driver if empty or unset
KernelDriver=nvidia
Module=nvidia
PMMethod=auto
# colon-separated path to the nvidia libraries
LibraryPath=/usr/lib/nvidia-bumblebee:/usr/lib32/nvidia-bumblebee
# comma-separated path of the directory containing nvidia_drv.so and the
# default Xorg modules path
XorgModulePath=/usr/lib/nvidia-bumblebee/xorg/,/usr/lib/xorg/modules
XorgConfFile=/etc/bumblebee/xorg.conf.nvidia
## Section with nouveau driver specific options, only parsed if Driver=nouveau
[driver-nouveau]
KernelDriver=nouveau
PMMethod=auto
XorgConfFile=/etc/bumblebee/xorg.conf.nouveau
[wicu@arch:~]$ cat /etc/bumblebee/xorg.conf.nvidia
Section "ServerLayout"
Identifier "Layout0"
Option "AutoAddDevices" "false"
EndSection
Section "Device"
Identifier "Device1"
Driver "nvidia"
VendorName "NVIDIA Corporation"
Option "NoLogo" "true"
Option "UseEDID" "false"
Option "ConnectedMonitor" "DFP"
EndSection

Failed to restart the CSSD during the interconnect failure

Hi all,
I run a small ATP on my LAB where i have
- 2x nodes RAC 11.2.0.2 & ASM (my OCR & Voting files are stored on ASM)
- 1 public interface <> eth0
- 1 private interface <> eth1
- 1 SCAN IP defined in the /etc/hosts file (i'm not using DNS or GNS)
The test i run was to shutdown the private interface (eth1) on node 1 and i saw that
1) all cluster services and cluster daemons on node 2 were killed and node 2 was evicted from the cluster by node 1
2) all new connections were redirected to the survived node
3) Oracle OHASD daemon was restarted on node 2 and tried to start the cluster services without success because private network between cluster nodes was down
Up to here everything worked as expected but once i turn on eth1 it took ~ 9 minutes for the CSSD to startup and bring all the components up & running.
The node2 alert logs showes
[ctssd(12949)]CRS-2402:The Cluster Time Synchronization Service aborted on host node2. Details at (:ctss_css_init1:) in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/ctssd/octssd.log.
2011-04-13 08:09:40.978
[ohasd(5058)]CRS-2765:Resource 'ora.cssd' has failed on server 'node2'.
2011-04-13 08:09:40.985
[/u01/oracle/installed/oracle_cluster-11.2.0.2-1/bin/oraagent.bin(5764)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/agent/ohasd/oraagent_oracle/oraagent_oracle.log";
2011-04-13 08:09:41.169
[ohasd(5058)]CRS-2765:Resource 'ora.asm' has failed on server 'node2'.
2011-04-13 08:09:50.337
[cssd(13103)]CRS-1713:CSSD daemon is started in clustered mode
2011-04-13 08:10:05.833
[cssd(13103)]CRS-1707:Lease acquisition for node node2 number 2 completed
2011-04-13 08:10:07.119
[cssd(13103)]CRS-1605:CSSD voting file is online: ORCL:CRS_DISK1_2G; details in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/cssd/ocssd.log.
2011-04-13 08:10:07.121
[cssd(13103)]CRS-1605:CSSD voting file is online: ORCL:CRS_DISK2_2G; details in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/cssd/ocssd.log.
2011-04-13 08:10:07.143
[cssd(13103)]CRS-1605:CSSD voting file is online: ORCL:CRS_DISK1_2G; details in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/cssd/ocssd.log.
2011-04-13 08:19:49.386
[/u01/oracle/installed/oracle_cluster-11.2.0.2-1/bin/cssdagent(13091)]CRS-5818:Aborted command 'start for resource: ora.cssd 1 1' for resource 'ora.cssd'. Details at (:CRSAGF00113:) {0:6:7} in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/agent/ohasd/oracssdagent_root/oracssdagent_root.log.
2011-04-13 08:19:49.387
[cssd(13103)]CRS-1656:The CSS daemon is terminating due to a fatal error; Details at (:CSSSC00012:) in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/cssd/ocssd.log
2011-04-13 08:19:49.387
[cssd(13103)]CRS-1603:CSSD on node node2 shutdown by user.
2011-04-13 08:19:54.501
[ohasd(5058)]CRS-2765:Resource 'ora.cssdmonitor' has failed on server 'node2'.
2011-04-13 08:19:57.723
[cssd(17068)]CRS-1713:CSSD daemon is started in clustered mode
2011-04-13 08:20:01.177
[ohasd(5058)]CRS-2765:Resource 'ora.diskmon' has failed on server 'node2'.
2011-04-13 08:20:13.167
[cssd(17068)]CRS-1707:Lease acquisition for node node2 number 2 completed pay attention at the timestamp 08:10:07.143 & 08:19:49.386
The error in the oracssdagent_root.log is
2011-04-13 08:09:49.286: [CLSFRAME][3014212592] New Framework state: 2
2011-04-13 08:09:49.286: [CLSFRAME][3014212592] M2M is starting...
2011-04-13 08:09:49.288: [ CRSCOMM][3014212592] Ipc: Starting send thread
2011-04-13 08:09:49.288: [ CRSCOMM][1092061504] Ipc: sendWork thread started.
2011-04-13 08:09:49.289: [ CRSCOMM][1105643840] IpcC: IPC Client thread started listening
2011-04-13 08:09:49.289: [ CRSCOMM][1105643840] IpcC: Received member number of 10
2011-04-13 08:09:49.290: [CLSFRAME][3014212592] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:node2
2011-04-13 08:09:49.290: [CLSFRAME][3014212592] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:node2
2011-04-13 08:09:49.291: [CLSFRAME][3014212592] Tints initialized with nodeId: 0 procId: 10
2011-04-13 08:09:49.291: [CLSFRAME][3014212592] Starting thread model named: MultiThread
2011-04-13 08:09:49.292: [CLSFRAME][3014212592] Starting thread model named: TimerSharedTM
2011-04-13 08:09:49.293: [CLSFRAME][3014212592] New Framework state: 3
2011-04-13 08:09:49.293: [    AGFW][3014212592] Agent Framework started successfully
2011-04-13 08:09:49.293: [    AGFW][1116150080] {0:10:2} Agfw engine module has enabled...
2011-04-13 08:09:49.293: [CLSFRAME][1116150080] {0:10:2} Module Enabling is complete
2011-04-13 08:09:49.293: [CLSFRAME][1116150080] {0:10:2} New Framework state: 6
2011-04-13 08:09:49.294: [CLSFRAME][3014212592] M2M is now powered by a doWork() thread.
2011-04-13 08:09:49.294: [    AGFW][1116150080] {0:10:2} Agent is started with userid: root , expected user: root
2011-04-13 08:09:49.294: [   AGENT][1116150080] {0:10:2} Static Version 11.2.0.2.0
2011-04-13 08:09:49.294: [    AGFW][1116150080] {0:10:2} Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:11
2011-04-13 08:09:49.302: [    AGFW][1116150080] {0:10:2} Agent received the message: RESTYPE_ADD[ora.cssd.type] ID 8196:12358
2011-04-13 08:09:49.302: [    AGFW][1116150080] {0:10:2} Added new restype: ora.cssd.type
2011-04-13 08:09:49.303: [    AGFW][1116150080] {0:10:2} Agent sending last reply for: RESTYPE_ADD[ora.cssd.type] ID 8196:12358
2011-04-13 08:09:49.305: [    AGFW][1116150080] {0:10:2} Agent received the message: RESOURCE_ADD[ora.cssd 1 1] ID 4356:12359
2011-04-13 08:09:49.305: [    AGFW][1116150080] {0:10:2} Added new resource: ora.cssd 1 1 to the agfw
2011-04-13 08:09:49.306: [    AGFW][1116150080] {0:10:2} Agent sending last reply for: RESOURCE_ADD[ora.cssd 1 1] ID 4356:12359
2011-04-13 08:09:49.308: [    AGFW][1116150080] {0:6:7} Agent received the message: RESOURCE_START[ora.cssd 1 1] ID 4098:12360
2011-04-13 08:09:49.308: [    AGFW][1116150080] {0:6:7} Preparing START command for: ora.cssd 1 1
2011-04-13 08:09:49.308: [    AGFW][1116150080] {0:6:7} ora.cssd 1 1 state changed from: UNKNOWN to: STARTING
2011-04-13 08:09:49.309: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_cssdstart: Start action called
2011-04-13 08:09:49.309: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_getattr: attr OMON_INITRATE, value 1000
2011-04-13 08:09:49.309: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_getattr: attr OMON_POLLRATE, value 500
2011-04-13 08:09:49.309: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_getattr: attr ORA_OPROCD_MODE, value
2011-04-13 08:09:49.310: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_getattr: attr PROCD_TIMEOUT, value 1000
2011-04-13 08:09:49.310: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_getattr: attr LOGGING_LEVEL, value 1
2011-04-13 08:09:49.310: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_cssdstart: loglevels CSSD=2,GIPCNM=2,GIPCGM=2,GIPCCM=2,CLSF=0,SKGFD=0,GPNP=1,OLR=0
2011-04-13 08:09:49.313: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_cssdstart: START action for resource /u01/oracle/installed/oracle_cluster-11.2.0.2-1/bin/ocssd: SUCCESS
2011-04-13 08:09:49.313: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_waitomon: start waiting
2011-04-13 08:09:49.313: [ CSSCLNT][1098377536]clsssInitNative: Init for agent
2011-04-13 08:09:50.317: [ CSSCLNT][1098377536]clsssInitNative: Init for agent
2011-04-13 08:09:51.319: [ CSSCLNT][1098377536]clsssInitNative: Init for agent
2011-04-13 08:09:51.322: [ CSSCLNT][1098377536]clssnsqueryfatal: css is fatal = 0
2011-04-13 08:09:51.322: [ USRTHRD][1098377536] clsncssd_thrdspawn: spawn OPROCD succ
2011-04-13 08:09:51.322: [ USRTHRD][1098377536] clsncssd_thrdspawn: spawn POLLMSG succ
2011-04-13 08:09:51.323: [ USRTHRD][1099954496] clsnpollmsg_main: starting pollmsg thread
2011-04-13 08:09:51.323: [ USRTHRD][1107745088] clsnproc_main: timeout of procd cannot be 0, now we set to default 1000.
2011-04-13 08:09:51.323: [ USRTHRD][1117727040] clsnwork_main: starting worker thread
2011-04-13 08:09:51.323: [ USRTHRD][1098377536] clsncssd_thrdspawn: spawn WORKER succ
2011-04-13 08:09:51.323: [ USRTHRD][1107745088] clsnproc_main: starting oprocd
2011-04-13 08:09:51.323: [ USRTHRD][1098377536] clsncssd_thrdspawn: spawn KILL succ
2011-04-13 08:10:07.151: [ USRTHRD][1098377536] clsnomon_init: css init done, nodenum 2
2011-04-13 08:10:07.151: [ USRTHRD][1098377536] clsnomon_WaitToRegister: waiting for first reconfiguration and kgzf initialization
2011-04-13 08:19:49.385: [CLSFRAME][3014212592] TM [MultiThread] is changing desired thread # to 3. Current # is 2
2011-04-13 08:19:49.387: [    AGFW][1111947584] {0:6:7} Created alert : (:CRSAGF00113:) : Aborting the command: start for resource: ora.cssd 1 1
2011-04-13 08:19:49.387: [ora.cssd][1111947584] {0:6:7} [start] clsncssd_cssdabort: sending shutdown abort to CSS with new ctx
2011-04-13 08:19:49.387: [ CSSCLNT][1098377536]clsssRecvMsg: wrong type request (0) on 0xc9 ret 0
2011-04-13 08:19:49.387: [ CSSCLNT][1098377536]clssnskgzfdone: RPC failed rc 1
2011-04-13 08:19:49.387: [ USRTHRD][1098377536] clsnomon_WaitToRegister: exadata initialization completed with rc=1
2011-04-13 08:19:49.387: [ USRTHRD][1098377536] clsnomon_init: problems in the CSS to allow OMON registration 2
2011-04-13 08:19:49.387: [ USRTHRD][1098377536] clsnomon_cleanup: to exit status = 2
2011-04-13 08:19:49.387: [ USRTHRD][1098377536] clsnomon_cleanup: failure, sending shutdown immediate to CSS
2011-04-13 08:19:49.387: [ USRTHRD][1098377536] CHECK action is in progress, Rejecting the check action requested by entry point for ora.cssd
2011-04-13 08:19:49.426: [    AGFW][2008402928] Starting the agent: /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/agent/ohasd/oracssdagent_root/
2011-04-13 08:19:49.426: [   AGENT][2008402928] Agent framework initialized, Process Id = 17013
2011-04-13 08:19:49.426: [ USRTHRD][2008402928] to enter agent main
2011-04-13 08:19:49.426: [ USRTHRD][2008402928] clsscssd_main: New soft limit for stack size is 1572864, hard limit is 4294967295
2011-04-13 08:19:49.434: [ USRTHRD][2008402928] clsncssd_main: setting priority to 4
2011-04-13 08:19:49.434: [ USRTHRD][2008402928] *** Agent Framework Started *** Do you have any idea why it took so long to bring all the components up & running?
Thanks a lot!!
G

Hi,
there is an internal timer for the clusterware ressources regarding restarting the ressources.
In case of a node eviction or clusterstack reboot the clusterware tries to startup again.
If the issue still persists, CRS will wait for some time to start the stack again. This "restart" try is based on a timer, which is set to 600 seconds (note this is not the ORA_CHECK_TIMEOUT) but the STARTUP_TIMEOUT.
Since a missing interconnect does have some implications (not only on the network but on the whole stack) it is expected, that the cluster does not start so fast automatically (because it still has the first start running.
There is even another "issue" connected to this - Oracle will only try several times (FAILURE_COUNT/FAILURE_THRESHOLD) to restart ressources. If he cannot restart cssd/crsd for several times, OCW will not try to startup automatically, but expects the administrator to solve the error and then startup again.
But actually this does make sense:
We have to give some time for an error to be resolved, before we start automatically. It does not matter if the restart of the node is delayed by this, because
=> If the error is fixed automatically, it will normally be fixed after a cluster/node reboot and hence cluster will come up
=> If the error is not fixed automatically, but manually, it can be expected that the administrator tells clusterware the issue is resolved. He does that by simply starting the stack (crsctl start crs)
=> If the error is fixed automaticall, but fixing took a while (lets say 15 minutes), it does not really matter if clusterware needs 10 more minutes to come up.
So what you see is expected, and wanted.
It would cost way too much to monitor all ressources regarding cluster problems and trigger a startup....
Sebastian

CSS Daemon error !!

Hello ,
I am having problems connecting from RMAN to the database (Oracle 10.2.0.3 running on CRS , AIX 5.3).
When i am trying to connect it shows following errors.
pr:/u00/oracle/product/10.2.0/crs/bin%*> rman target /
Recovery Manager: Release 10.2.0.3.0 - Production on Mon Oct 13 17:09:06 2008
Copyright (c) 1982, 2005, Oracle. All rights reserved.
RMAN-06900: WARNING: unable to generate V$RMAN_STATUS or V$RMAN_OUTPUT row
RMAN-06901: WARNING: disabling update of the V$RMAN_STATUS and V$RMAN_OUTPUT rows
ORACLE error from target database:
ORA-29701: unable to connect to Cluster Manager
connected to target database: PSP (DBID=695888555)
== Then i checked CSSD which shows like below
pr://u00/oracle/product/10.2.0/crs/bin:/crsctl check crs
Failure 1 contacting CSS daemon
ON 08th Oct , system admin added one more port and i think they cloned from existing (PR) to (PR-dr) from Smitty.. I think that time CRS switched from PR to PR-dr while system admins changed back the configuration.
pr:/u00/oracle/product/10.2.0/crs/log%*> ls -lrt
total 0
drwxrwx--- 2 oracle dba 256 Mar 06 2008 crs
drwxr-xr-t 8 root dba 256 Mar 06 2008 pr
drwxr-xr-x 3 root system 256 Oct 08 14:00 pr-dr
pr:/u00/oracle/product/10.2.0/crs/log/PR-dr/racg ls -lrt
drwxr-xr-x 2 root system 256 Oct 08 14:00 racgmain
drwxr-xr-x 2 root system 256 Oct 08 14:00 racgeut
-rw-r--r-- 1 root system 1737625 Oct 13 17:15 ora.pr.vip.log
--- tail ora.pra.vip.log
2008-10-13 17:13:53.231: [ RACG][1] [802942][1][ora.pr.vip]: clsrccssgetctx: clsssinit() failed. rc=21
2008-10-13 17:13:53.264: [ RACG][1] [802942][1][ora.pr.vip]: clsrcgetprsrctx: prsr_init_ext returned rc = 5
I think if i will stop the crs on this node while crs will be active on other node .. Will this work ?
Or can i use crsstart to restart cssd
Or i can use localconfig
Please help.
Thanks
Edited by: P explorer on Oct 13, 2008 5:22 PM

Hi P explorer
Sorry for the late answer, may this work-around should help you out of that issue. it applies to Version: 10.1.0.2 to 10.2.0.4
Cause
The hidden directory '/var/tmp/.oracle' was removed while instances & the CRS stack were up and running. Typically this directory contains a number of "special" socket files that are used by local clients to connect via the IPC protocol (sqlnet) to various Oracle processes including the TNS listener, the CSS, CRS & EVM daemons or even the database instance. These files are created when the "listening" process starts.
Solution
The only way to re-create these special files is to restart (instance, listener, CRS). In a RAC environment this requires the shutdown & restart of the entire CRS stack.
As these special files are required to communicate with the various CRS daemons, it most likely will not be possible to stop (and restart) the CRS stack using the following commands as user root - but it won't hurt to try it anyway:
10gR1:
# /etc/init.d./init.crs stop
# /etc/init.d./init.crs start
10gR2:
# $ORA_CRS_HOME/bin/crsctl stop crs
# $ORA_CRS_HOME/bin/crsctl start crs
If the above fails to successfully stop the CRS stack, a system reboot will be inevitable.
As for deleting files from temporary directory via a cronjob:
the directory '/var/tmp/.oracle' (on some platform /tmp/.oracle) should be excluded from such jobs. The files in this directory on occupy a few bytes and generally do not need to be cleaned up.
A good practice for deleting files is to check if the file is still being opened by a process, most Unix platforms have a 'fuser' command which returns the process IDs of such processes.
Good luck.

CRS-0184: Cannot communicate with the CRS daemon.

I am getting teh foll error when starting up a database
[oracle@linux2 ~]$ srvctl start database -d orcl
PRKH-1010 : Unable to communicate with CRS services.
[oracle@linux2 ~]$ product/crs/bin/crs_start -all
CRS-0184: Cannot communicate with the CRS daemon.
From the available information on the net, I have already tried the below steps.
a) Deleted all the files from /var/tmp/.oracle
b) Ensured that the deamon is running
[oracle@linux2 ~]$ ps -ef | grep crs
root 6192 1 0 15:31 ? 00:00:00 /bin/sh /etc/init.d/init.crsd run
oracle 1284 9630 0 16:02 pts/1 00:00:00 grep crs
c) The foll lines are already added to the /etc/inittab file
h1:3:respawn:/sbin/init.d/init.evmd run >/dev/null 2>&1 </dev/null
h2:3:respawn:/sbin/init.d/init.cssd fatal >/dev/null 2>&1 </dev/null
h3:3:respawn:/sbin/init.d/init.crsd run >/dev/null 2>&1 </dev/null
Plz help. I'm still getting the error on one node of the Cluster.

[oracle@linux2 bin]$ crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM
The node is setup on RHEL 4u7 (2.6.9-78.0.13.ELsmp)
The first error in the crsd.log file says
2009-11-05 06:07:57.196: [ CRSOCR][3086907072]0OCR api procr_open_key failed for key CRS.CUR. OCR error code = 4 OCR error msg: PROC-4: The cluster registry key to be operated on does not exist.
Also, I have the '/u02/oradata/orcl' for the OCR and the CSS Files. On thsi node this directory is empty.
Whereas on the other node in the sam cluster, thsi directory contains the OCR and CSS files and their mirrors.
Is my OCR corrupted on thsi node? How can I restore it??

OCR Problam init.cssd startcheck

Hi
After installing OCR it not run properly.
Follow some old post I tried some action – here the output
But I do not know what to do forward to fixed it ?
Some more question :
Way raw1 and raw2 permission is for user root and not oracle(by doing it I follow one of the guide) ?
What do the commend “dd if=/dev/zero of=votingdisk bs=1M count=256”
*[root@rac1 bin]# ps -ef|grep init.d*
root 4996 1 0 00:07 ? 00:00:00 /bin/sh /etc/init.d/init.evmd run
root 4997 1 0 00:07 ? 00:00:00 /bin/sh /etc/init.d/init.cssd fatal
root 4998 1 0 00:07 ? 00:00:00 /bin/sh /etc/init.d/init.crsd run
root 5044 4997 0 00:07 ? 00:00:00 /bin/sh /etc/init.d/init.cssd startcheck
root 5081 4996 0 00:07 ? 00:00:00 /bin/sh /etc/init.d/init.cssd startcheck
root 5237 4998 0 00:07 ? 00:00:00 /bin/sh /etc/init.d/init.cssd startcheck
root 25331 9511 0 09:37 pts/2 00:00:00 grep init.d
*[root@rac1 bin]# /etc/init.d/rawdevices status*
-bash: /etc/init.d/rawdevices: No such file or directory
*[root@rac1 bin]# ./crsctl check crs*
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM
*[root@rac1 bin]# less /var/log/messages*
*[root@rac1 bin]# cat /tmp/crsctl.5081*
Failure -2 opening file handle for (raw2)
Failure 1 checking the CSS voting disk 'raw2'.
Not able to read adequate number of voting disks
*[root@rac1 bin]# cat /etc/oracle/ocr.loc*
ocrconfig_loc=/dev/raw/raw1
local_only=FALSE
*[root@rac1 bin]# ./crsctl query css votedisk*
0. 0 /dev/raw/raw2
located 1 votedisk(s).
*[root@rac1 bin]# ls -l /dev/raw*
total 0
crw-r----- 1 root oinstall 162, 1 Aug 18 11:21 raw1
crw-r----- 1 root oinstall 162, 2 Aug 18 11:21 raw2
crw-r--r-- 1 oracle oinstall 162, 3 Aug 18 11:21 raw3
crw-r--r-- 1 oracle oinstall 162, 4 Aug 18 11:21 raw4
crw-r--r-- 1 oracle oinstall 162, 5 Aug 18 11:21 raw5
*[root@rac1 bin]# raw -qa*
/dev/raw/raw1: bound to major 8, minor 17
/dev/raw/raw2: bound to major 8, minor 33
/dev/raw/raw3: bound to major 8, minor 49
/dev/raw/raw4: bound to major 8, minor 65
/dev/raw/raw5: bound to major 8, minor 81
AS Oracle User
*[oracle@rac1 crs]$ dd if=/dev/zero of=votingdisk bs=1M count=256*
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 3.86209 seconds, 69.5 MB/s

refer:
http://oracleinstance.blogspot.com/2010/08/recover-corruptmissing-ocr-and-voting.html
http://download.oracle.com/docs/cd/B19306_01/rac.102/b28759/adminoc.htm#BJFJAJCI
if you metalink access, please refer
399482.1 How to recreate OCR/Voting disk accidentally deleted
358620.1 How to recreate OCR/Voting disk in 10gR1/R2 RAC
279793.1 How to Restore a Lost Voting Disk in 10g
source:http://www.juliandyke.com/References/Clusterware.html
Edited by: rajeysh on Aug 19, 2011 8:00 PM

Failure at final check of Oracle CRS stack. 10 on the first node.

Hi everyone
I trying to install an Oracle RAC 10gr2 on an Oracle Enterprise Linux AS release 4 (October Update 7) , but I'm having this problem
root@fporn01 crs# ./root.sh
Checking to see if Oracle CRS stack is already configured
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
assigning default hostname fporn01 for node 1.
assigning default hostname fporn02 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: fporn01 fporn01-priv fporn01
node 2: fporn02 fporn02-priv fporn02
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Failure at final check of Oracle CRS stack.
+10+
forget about the node names!!!!
but on the second node everything went fine, so I'm sure this is not a connectivity issue.
the iptables service is stopped and disabled
check the results after running the root.sh script
root@fporn02 ~# /u01/app/crs/root.sh
Checking to see if Oracle CRS stack is already configured
+/etc/oracle does not exist. Creating it now.+
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
assigning default hostname fporn01 for node 1.
assigning default hostname fporn02 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: fporn01 fporn01-priv fporn01
node 2: fporn02 fporn02-priv fporn02
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
fporn02
CSS is inactive on these nodes.
fporn01
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.
this is the log of crs on the first node
root@fporn01 bin# cat /u01/app/crs/log/fporn01/alertfporn01.log
+2009-06-24 17:27:37.695+
client(9045)CRS-1006:The OCR location /u02/oradata/orcl/OCRFile_mirror is inaccessible. Details in /u01/app/crs/log/fporn01/client/ocrconfig_9045.log.
+2009-06-24 17:27:37.741+
client(9045)CRS-1001:The OCR was formatted using version 2.
+2009-06-24 17:28:24.544+
client(9092)CRS-1801:Cluster pdb-rac configured with nodes fporn01 fporn02 .
this is the log of crs on the second node
root@fporn02 ~# cat /u01/app/crs/log/fporn02/alertfporn02.log
+2009-06-24 18:09:09.307+
cssd(16991)CRS-1605:CSSD voting file is online: /u02/oradata/orcl/CSSFile. Details in /u01/app/crs/log/fporn02/cssd/ocssd.log.
+2009-06-24 18:09:09.307+
cssd(16991)CRS-1605:CSSD voting file is online: /u02/oradata/orcl/CSSFile_mirror1. Details in /u01/app/crs/log/fporn02/cssd/ocssd.log.
+2009-06-24 18:09:09.310+
cssd(16991)CRS-1605:CSSD voting file is online: /u02/oradata/orcl/CSSFile_mirror2. Details in /u01/app/crs/log/fporn02/cssd/ocssd.log.
+2009-06-24 18:09:12.441+
cssd(16991)CRS-1601:CSSD Reconfiguration complete. Active nodes are fporn02 .
I have rechecked the Remote Access / User Equivalence
after run the OCRCHECK command ia have this information
root@fporn01 bin# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 262144
Used space (kbytes) : 312
Available space (kbytes) : 261832
ID : 255880615
Device/File Name : /u02/oradata/orcl/OCRFile
Device/File integrity check succeeded
Device/File Name : /u02/oradata/orcl/OCRFile_mirror
Device/File integrity check succeeded
Cluster registry integrity check succeeded
on the second node i get the same output
root@fporn02 bin# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 262144
Used space (kbytes) : 312
Available space (kbytes) : 261832
ID : 255880615
Device/File Name : /u02/oradata/orcl/OCRFile
Device/File integrity check succeeded
Device/File Name : /u02/oradata/orcl/OCRFile_mirror
Device/File integrity check succeeded
Cluster registry integrity check succeeded
I have reviewed the following metalink notes but none of them seems to solve my problem
*344994.1*
*240001.1*
*725878.1*
*329450.1*
*734221.1*
I have done a research trough many forums, but always the fail is on the second node, but my fail is on the first node.
I hope anyone could help me.
this is the output of cluvfy
Performing pre-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "fporn01"
Destination Node Reachable?
fporn01 yes
fporn02 yes
Result: Node reachability check passed from node "fporn01".
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Comment
fporn02 passed
fporn01 passed
Result: User equivalence check passed for user "oracle".
Checking administrative privileges...
Check: Existence of user "oracle"
Node Name User Exists Comment
fporn02 yes passed
fporn01 yes passed
Result: User existence check passed for "oracle".
Check: Existence of group "oinstall"
Node Name Status Group ID
fporn02 exists 501
fporn01 exists 501
Result: Group existence check passed for "oinstall".
Check: Membership of user "oracle" in group "oinstall" as Primary
Node Name User Exists Group Exists User in Group Primary Comment
fporn02 yes yes yes yes passed
fporn01 yes yes yes yes passed
Result: Membership check for user "oracle" in group "oinstall" as Primary passed.
Administrative privileges check passed.
Checking node connectivity...
Interface information for node "fporn02"
Interface Name IP Address Subnet
eth0 10.218.108.245 10.218.108.0
eth1 192.168.1.2 192.168.1.0
Interface information for node "fporn01"
Interface Name IP Address Subnet
eth0 10.218.108.244 10.218.108.0
eth1 192.168.1.1 192.168.1.0
eth2 172.16.9.210 172.16.9.0
Check: Node connectivity of subnet "10.218.108.0"
Source Destination Connected?
fporn02:eth0 fporn01:eth0 yes
Result: Node connectivity check passed for subnet "10.218.108.0" with node(s) fporn02,fporn01.
Check: Node connectivity of subnet "192.168.1.0"
Source Destination Connected?
fporn02:eth1 fporn01:eth1 yes
Result: Node connectivity check passed for subnet "192.168.1.0" with node(s) fporn02,fporn01.
Check: Node connectivity of subnet "172.16.9.0"
Result: Node connectivity check passed for subnet "172.16.9.0" with node(s) fporn01.
Suitable interfaces for the private interconnect on subnet "10.218.108.0":
fporn02 eth0:10.218.108.245
fporn01 eth0:10.218.108.244
Suitable interfaces for the private interconnect on subnet "192.168.1.0":
fporn02 eth1:192.168.1.2
fporn01 eth1:192.168.1.1
ERROR:
Could not find a suitable set of interfaces for VIPs.
Result: Node connectivity check failed.
Checking system requirements for 'crs'...
Check: Total memory
Node Name Available Required Comment
fporn02 7.93GB (8310276KB) 512MB (524288KB) passed
fporn01 7.93GB (8310276KB) 512MB (524288KB) passed
Result: Total memory check passed.
Check: Free disk space in "/tmp" dir
Node Name Available Required Comment
fporn02 9.57GB (10037300KB) 400MB (409600KB) passed
fporn01 9.55GB (10012168KB) 400MB (409600KB) passed
Result: Free disk space check passed.
Check: Swap space
Node Name Available Required Comment
fporn02 8.81GB (9240568KB) 1GB (1048576KB) passed
fporn01 8.81GB (9240568KB) 1GB (1048576KB) passed
Result: Swap space check passed.
Check: System architecture
Node Name Available Required Comment
fporn02 i686 i686 passed
fporn01 i686 i686 passed
Result: System architecture check passed.
Check: Kernel version
Node Name Available Required Comment
fporn02 2.6.9-78.0.0.0.1.ELhugemem 2.4.21-15EL passed
fporn01 2.6.9-78.0.0.0.1.ELhugemem 2.4.21-15EL passed
Result: Kernel version check passed.
Check: Package existence for "make-3.79"
Node Name Status Comment
fporn02 make-3.80-7.EL4 passed
fporn01 make-3.80-7.EL4 passed
Result: Package existence check passed for "make-3.79".
Check: Package existence for "binutils-2.14"
Node Name Status Comment
fporn02 binutils-2.15.92.0.2-25 passed
fporn01 binutils-2.15.92.0.2-25 passed
Result: Package existence check passed for "binutils-2.14".
Check: Package existence for "gcc-3.2"
Node Name Status Comment
fporn02 gcc-3.4.6-10.0.1 passed
fporn01 gcc-3.4.6-10.0.1 passed
Result: Package existence check passed for "gcc-3.2".
Check: Package existence for "glibc-2.3.2-95.27"
Node Name Status Comment
fporn02 glibc-2.3.4-2.41 passed
fporn01 glibc-2.3.4-2.41 passed
Result: Package existence check passed for "glibc-2.3.2-95.27".
Check: Package existence for "compat-db-4.0.14-5"
Node Name Status Comment
fporn02 compat-db-4.1.25-9 passed
fporn01 compat-db-4.1.25-9 passed
Result: Package existence check passed for "compat-db-4.0.14-5".
Check: Package existence for "compat-gcc-7.3-2.96.128"
Node Name Status Comment
fporn02 missing failed
fporn01 missing failed
Result: Package existence check failed for "compat-gcc-7.3-2.96.128".
++Check: Package existence for "compat-gcc-c++-7.3-2.96.128"++
Node Name Status Comment
fporn02 missing failed
fporn01 missing failed
++Result: Package existence check failed for "compat-gcc-c++-7.3-2.96.128".++
++Check: Package existence for "compat-libstdc++-7.3-2.96.128"++
Node Name Status Comment
fporn02 missing failed
fporn01 missing failed
++Result: Package existence check failed for "compat-libstdc++-7.3-2.96.128".++
++Check: Package existence for "compat-libstdc++-devel-7.3-2.96.128"++
Node Name Status Comment
fporn02 missing failed
fporn01 missing failed
++Result: Package existence check failed for "compat-libstdc++-devel-7.3-2.96.128".++
Check: Package existence for "openmotif-2.2.3"
Node Name Status Comment
fporn02 openmotif-2.2.3-10.2.el4 passed
fporn01 openmotif-2.2.3-10.2.el4 passed
Result: Package existence check passed for "openmotif-2.2.3".
Check: Package existence for "setarch-1.3-1"
Node Name Status Comment
fporn02 setarch-1.6-1 passed
fporn01 setarch-1.6-1 passed
Result: Package existence check passed for "setarch-1.3-1".
Check: Group existence for "dba"
Node Name Status Comment
fporn02 exists passed
fporn01 exists passed
Result: Group existence check passed for "dba".
Check: Group existence for "oinstall"
Node Name Status Comment
fporn02 exists passed
fporn01 exists passed
Result: Group existence check passed for "oinstall".
Check: User existence for "nobody"
Node Name Status Comment
fporn02 exists passed
fporn01 exists passed
Result: User existence check passed for "nobody".
System requirement failed for 'crs'
Pre-check for cluster services setup was unsuccessful on all the nodes.

forget about my last post, it was my mistake, I rebooted the server and the clustered file system service did not start up at boot time.
sorry
this is what I really got in /var/log/messages
after manually running crs daemons
Jun 26 16:43:07 fporn01 su(pam_unix)[10020]: session opened for user oracle by (uid=0)
Jun 26 16:43:07 fporn01 su(pam_unix)[10020]: session closed for user oracle
Jun 26 16:43:07 fporn01 logger: Cluster Ready Services completed waiting on dependencies.
Jun 26 16:44:07 fporn01 su(pam_unix)[9977]: session opened for user oracle by (uid=0)
Jun 26 16:45:31 fporn01 su(pam_unix)[10293]: session opened for user oracle by (uid=0)
Jun 26 16:45:32 fporn01 su(pam_unix)[10293]: session closed for user oracle
Jun 26 16:45:32 fporn01 logger: Cluster Ready Services completed waiting on dependencies.
Jun 26 16:45:40 fporn01 su(pam_unix)[10351]: session opened for user oracle by (uid=0)
Jun 26 16:45:40 fporn01 su(pam_unix)[10351]: session closed for user oracle
Jun 26 16:45:40 fporn01 su(pam_unix)[10415]: session opened for user oracle by (uid=0)
Jun 26 16:45:40 fporn01 su(pam_unix)[10415]: session closed for user oracle
Jun 26 16:45:40 fporn01 logger: Cluster Ready Services completed waiting on dependencies.
Jun 26 16:46:32 fporn01 su(pam_unix)[10591]: session opened for user oracle by (uid=0)
Jun 26 16:46:40 fporn01 logger: Running CRSD with TZ =
after running ps -ef | grep -E 'init|d.bin|ocls|oprocd|diskmon|evmlogger|PID'
[root@fporn01 ~]# ps -ef | grep -E 'init|d.bin|ocls|oprocd|diskmon|evmlogger|PID'
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 15:33 ? 00:00:00 init [5]
root 9869 7951 0 16:40 pts/1 00:00:00 [init.crsd] <defunct>
oracle 10053 9977 0 16:44 ? 00:00:00 /u01/app/crs/bin/evmd.bin
root 10249 7951 0 16:45 pts/1 00:00:00 /bin/sh /etc/init.d/init.cssd fatal
root 10341 7951 0 16:45 pts/1 00:00:00 /u01/app/crs/bin/crsd.bin reboot
root 10551 10249 0 16:46 pts/1 00:00:00 /bin/sh /etc/init.d/init.cssd daemon
oracle 10618 10592 0 16:46 ? 00:00:00 /u01/app/crs/bin/ocssd.bin
oracle 10926 10053 0 16:46 ? 00:00:00 /u01/app/crs/bin/evmlogger.bin -o /u01/app/crs/evm/log/evmlogger.info -l /u01/app/crs/evm/log/evmlogger.log
root 16658 9461 0 16:50 pts/2 00:00:00 grep -E init|d.bin|ocls|oprocd|diskmon|evmlogger|PID
CRS daemons finally work
*but i get this error when i run [oracle@fporn01 cluvfy]$ ./runcluvfy.sh stage -post crsinst -n fporn01,fporn02 -verbose*
Performing post-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "fporn01"
Destination Node                      Reachable?
fporn01                               yes
fporn02                               yes
Result: Node reachability check passed from node "fporn01".
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name                             Comment
fporn02                               passed
fporn01                               passed
Result: User equivalence check passed for user "oracle".
ERROR:
CRS is not installed on any of the nodes.
Verification cannot proceed.
Post-check for cluster services setup was unsuccessful on all the nodes.

Grid on a standalone server Cluster daemons not starting during server boot

Hi!I have installed Oracle Grid on a standalone server and setup Oracle db 11.2.0.2 on Oracle Linux 6.2 64 bit server.When I reoot the server and run crs_stat -t,several daemons havent started thus the ASM and db instances are also down as below
Name Type Target State Host
ora.DATA.dg ora....up.type OFFLINE OFFLINE
ora.FRADG.dg ora....up.type OFFLINE OFFLINE
ora....ER.lsnr ora....er.type ONLINE ONLINE amldb01dc
ora.amldb.db ora....se.type OFFLINE OFFLINE
ora.asm ora.asm.type OFFLINE OFFLINE
ora.cssd ora.cssd.type ONLINE OFFLINE
ora.diskmon ora....on.type ONLINE OFFLINE
ora.evmd ora.evm.type ONLINE ONLINE amldb01dc
ora.ons ora.ons.type OFFLINE OFFLINE
I am forced to manually start the daemons via command crsctl start resource -all then I manually start the ASM and db instances.
Yet when I run the commands
crsctl config has
CRS-4622: Oracle High Availability Services autostart is enabled.
crsctl check has
CRS-4638: Oracle High Availability Services is online.
Thus I would assume the daemons would start automatically during boot.
How can I resolve this?

crstcl check crs seems not to be applicable to single instance but for RAC,since I have a single instance db,the output of the command is as below
crsctl check crs
Parse error:
'crs' is an invalid argument
Brief usage:
crsctl check has
Check status of OHAS.
There is no crsd.log in path /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/crsd.The output of crs_stat -t is already posted in initial post and output of entire alert log is as below
2012-07-03 16:18:35.975
[client(8182)]CRS-2101:The OLR was formatted using version 3.
2012-07-03 16:18:36.471
[client(8218)]CRS-1001:The OCR was formatted using version 3.
[client(8293)]CRS-10001:CRS-6021: No msg for has:crs-6021 [l][unlimited]
[client(8294)]CRS-10001:CRS-6021: No msg for has:crs-6021 [n][65536]
2012-07-03 16:18:42.528
[ohasd(8291)]CRS-2112:The OLR service started on node amldb01dc.
2012-07-03 16:18:42.538
[ohasd(8291)]CRS-1301:Oracle High Availability Service started on node amldb01dc.
2012-07-03 16:18:51.271
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(8465)]CRS-5815:Agent '/u01/app/oracle/product/11.2.0/grid/bin/oraagent_grid' could not find any base type entry points for type 'ora.daemon.type'. Details at (:CRSAGF00108:) {0:1:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-03 16:18:52.299
[evmd(8479)]CRS-1401:EVMD started on node amldb01dc.
[client(8520)]CRS-10001:03-Jul-12 16:18 ACFS-9459: ADVM/ACFS is not supported on this OS version: 'error: file /etc/SuSE-release: No such file or directory
[client(8889)]CRS-10001:03-Jul-12 16:19 ACFS-9459: ADVM/ACFS is not supported on this OS version: 'error: file /etc/SuSE-release: No such file or directory
2012-07-03 16:19:22.600
[cssd(9014)]CRS-1713:CSSD daemon is started in local-only mode
2012-07-03 16:19:31.128
[cssd(9014)]CRS-1601:CSSD Reconfiguration complete. Active nodes are amldb01dc .
[client(10038)]CRS-10001:03-Jul-12 16:25 ACFS-9459: ADVM/ACFS is not supported on this OS version: 'error: file /etc/SuSE-release: No such file or directory
2012-07-03 16:34:13.791
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(8465)]CRS-5010:Update of configuration file "/u01/app/oracle/product/11.2.0/dbhome_1/srvm/admin/oratab.bak.amldb01dc" failed: details at "(:CLSN00011:)" in "/u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log"
2012-07-03 16:37:39.674
[ohasd(8291)]CRS-2765:Resource 'ora.DATA.dg' has failed on server 'amldb01dc'.
2012-07-03 16:37:43.636
[ohasd(8291)]CRS-2767:Resource state recovery not attempted for 'ora.asm' as its target state is OFFLINE
2012-07-03 16:38:06.905
[cssd(9014)]CRS-1603:CSSD on node amldb01dc shutdown by user.
2012-07-03 16:38:07.006
[cssd(9014)]CRS-1660:The CSS daemon shutdown has completed
2012-07-03 16:38:10.118
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(8465)]CRS-5016:Process "/u01/app/oracle/product/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log"
2012-07-03 16:42:09.578
[ohasd(4827)]CRS-2112:The OLR service started on node amldb01dc.
2012-07-03 16:42:09.649
[ohasd(4827)]CRS-1301:Oracle High Availability Service started on node amldb01dc.
2012-07-03 16:42:10.437
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4992)]CRS-5815:Agent '/u01/app/oracle/product/11.2.0/grid/bin/oraagent_grid' could not find any base type entry points for type 'ora.daemon.type'. Details at (:CRSAGF00108:) {0:2:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-03 16:42:10.667
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4992)]CRS-5016:Process "/u01/app/oracle/product/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log"
2012-07-03 16:42:40.463
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4992)]CRS-5818:Aborted command 'check for resource: ora.DATA.dg amldb01dc 1' for resource 'ora.DATA.dg'. Details at (:CRSAGF00113:) {0:0:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-03 16:42:40.464
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4992)]CRS-5818:Aborted command 'check for resource: ora.FRADG.dg amldb01dc 1' for resource 'ora.FRADG.dg'. Details at (:CRSAGF00113:) {0:0:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-03 16:42:54.641
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(5046)]CRS-5815:Agent '/u01/app/oracle/product/11.2.0/grid/bin/oraagent_grid' could not find any base type entry points for type 'ora.daemon.type'. Details at (:CRSAGF00108:) {0:4:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-03 16:42:55.675
[evmd(5060)]CRS-1401:EVMD started on node amldb01dc.
2012-07-03 16:54:35.840
[cssd(5256)]CRS-1713:CSSD daemon is started in local-only mode
2012-07-03 16:54:44.396
[cssd(5256)]CRS-1601:CSSD Reconfiguration complete. Active nodes are amldb01dc .
2012-07-03 16:54:56.832
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(5046)]CRS-5010:Update of configuration file "/u01/app/oracle/product/11.2.0/dbhome_1/srvm/admin/oratab.bak.amldb01dc" failed: details at "(:CLSN00011:)" in "/u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log"
2012-07-04 10:03:53.530
[ohasd(4827)]CRS-2767:Resource state recovery not attempted for 'ora.asm' as its target state is OFFLINE
2012-07-04 10:04:02.701
[cssd(5256)]CRS-1603:CSSD on node amldb01dc shutdown by user.
2012-07-04 10:04:02.809
[cssd(5256)]CRS-1660:The CSS daemon shutdown has completed
2012-07-04 10:04:03.008
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(5046)]CRS-5016:Process "/u01/app/oracle/product/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log"
2012-07-04 10:07:48.492
[ohasd(4698)]CRS-2112:The OLR service started on node amldb01dc.
2012-07-04 10:07:48.534
[ohasd(4698)]CRS-1301:Oracle High Availability Service started on node amldb01dc.
2012-07-04 10:07:49.059
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4946)]CRS-5815:Agent '/u01/app/oracle/product/11.2.0/grid/bin/oraagent_grid' could not find any base type entry points for type 'ora.daemon.type'. Details at (:CRSAGF00108:) {0:1:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:07:49.192
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4946)]CRS-5016:Process "/u01/app/oracle/product/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log"
2012-07-04 10:08:19.085
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4946)]CRS-5818:Aborted command 'check for resource: ora.DATA.dg amldb01dc 1' for resource 'ora.DATA.dg'. Details at (:CRSAGF00113:) {0:0:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:08:19.093
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4946)]CRS-5818:Aborted command 'check for resource: ora.FRADG.dg amldb01dc 1' for resource 'ora.FRADG.dg'. Details at (:CRSAGF00113:) {0:0:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:08:33.278
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(5053)]CRS-5815:Agent '/u01/app/oracle/product/11.2.0/grid/bin/oraagent_grid' could not find any base type entry points for type 'ora.daemon.type'. Details at (:CRSAGF00108:) {0:4:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:08:34.319
[evmd(5069)]CRS-1401:EVMD started on node amldb01dc.
2012-07-04 10:27:13.716
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(5053)]CRS-5016:Process "/u01/app/oracle/product/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log"
2012-07-04 10:30:57.260
[ohasd(4619)]CRS-2112:The OLR service started on node amldb01dc.
2012-07-04 10:30:57.280
[ohasd(4619)]CRS-1301:Oracle High Availability Service started on node amldb01dc.
2012-07-04 10:30:57.660
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4992)]CRS-5815:Agent '/u01/app/oracle/product/11.2.0/grid/bin/oraagent_grid' could not find any base type entry points for type 'ora.daemon.type'. Details at (:CRSAGF00108:) {0:1:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:30:57.784
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4992)]CRS-5016:Process "/u01/app/oracle/product/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log"
2012-07-04 10:31:27.685
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4992)]CRS-5818:Aborted command 'check for resource: ora.FRADG.dg amldb01dc 1' for resource 'ora.FRADG.dg'. Details at (:CRSAGF00113:) {0:0:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:31:27.685
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4992)]CRS-5818:Aborted command 'check for resource: ora.DATA.dg amldb01dc 1' for resource 'ora.DATA.dg'. Details at (:CRSAGF00113:) {0:0:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:31:41.868
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(5168)]CRS-5815:Agent '/u01/app/oracle/product/11.2.0/grid/bin/oraagent_grid' could not find any base type entry points for type 'ora.daemon.type'. Details at (:CRSAGF00108:) {0:4:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:31:42.916
[evmd(5184)]CRS-1401:EVMD started on node amldb01dc.
2012-07-04 10:39:17.166
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(5168)]CRS-5016:Process "/u01/app/oracle/product/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log"
2012-07-04 10:43:01.219
[ohasd(4575)]CRS-2112:The OLR service started on node amldb01dc.
2012-07-04 10:43:01.240
[ohasd(4575)]CRS-1301:Oracle High Availability Service started on node amldb01dc.
2012-07-04 10:43:01.643
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4971)]CRS-5815:Agent '/u01/app/oracle/product/11.2.0/grid/bin/oraagent_grid' could not find any base type entry points for type 'ora.daemon.type'. Details at (:CRSAGF00108:) {0:1:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:43:01.781
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4971)]CRS-5016:Process "/u01/app/oracle/product/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log"
2012-07-04 10:43:31.680
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4971)]CRS-5818:Aborted command 'check for resource: ora.FRADG.dg amldb01dc 1' for resource 'ora.FRADG.dg'. Details at (:CRSAGF00113:) {0:0:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:43:31.680
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(4971)]CRS-5818:Aborted command 'check for resource: ora.DATA.dg amldb01dc 1' for resource 'ora.DATA.dg'. Details at (:CRSAGF00113:) {0:0:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:43:45.864
[u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin(5120)]CRS-5815:Agent '/u01/app/oracle/product/11.2.0/grid/bin/oraagent_grid' could not find any base type entry points for type 'ora.daemon.type'. Details at (:CRSAGF00108:) {0:4:2} in /u01/app/oracle/product/11.2.0/grid/log/amldb01dc/agent/ohasd/oraagent_grid/oraagent_grid.log.
2012-07-04 10:43:46.915
[evmd(5136)]CRS-1401:EVMD started on node amldb01dc.

ASM : CSSD

Hi,
I am reading oracle 10g Rac-grid-service,Murali vallath book.
The page number 125 says that " When installing ASM on a single-instance configuration, ensure that the cluster synchronization services (CSS) module is installed."
Does it mean voting disk is mandatory for a single instance database.
and
Page number 48 says that "
Using a vacuous monitoring over the voting disk locations, CSS
performs state changes to bring the voting disk online. This is to
determine if CSS has a registered MASTER node already active. The
various states of the voting disk are
1 - Not configured and no thread has been spawned
2 - Threads are spawned
3 - Thread started and disk is offline
4 - The voting disk is online "
what is vacuous monitoring?
Is there any way to identify voting disk state?
Please help me.
Thanks & Regards,

If you are installing single instance ASM then CSSD daemon is used to communicate with ASM disks. In single instance environment, Voting disk is not needed. Instead, CSS will communicate with OLR which is oracle local registry. This daemon is mandatory to start ASM instance for non RAC environment. If you kill this daemon process then node will reboot so please be careful with this process.

Acrobat Pro XI install failure

When I attempt to install Acrobat XI Pro from DVD, I get a message, "installation failure, contact software manufacture". I am on fairly new iMac that meets all specs.

attach a screenshot of the error message or download a trial and activate with your serial number, http://www.adobe.com/cfusion/tdrc/index.cfm?product=acrobat_pro

E72 Start-up failure after nokia logo ;(

all..help me to fix this problems..
My E72 White edition Z-white, i buy 2 month ago
( V.031.023 , 31 march 2010)
Made in China , don’t know it’s original or just fake...
Slow reponse and always restart itself, the problems is :
1. when i open a website its sometimes error and restart itself..
2. the response when open an images or video and other applications is
      very slow.., and sometimes make it restart itself...
3. now the huge problem is totally can't using my phone..
     when turn on my phone after nokia logo appear :
     “phone start-Up failure, contact the retailer “
     what the hell, retailer can fix this poblem?
     Im try to remove the memory card
     (4GB micrSD original from phone packages) then put it back again...
     And it’s work, but in few hours, error message is appear back...so sad..
Please help me to solve this problem : (

/t5/Pool-of-Knowledge/Avoiding-fake-Nokia-phones/td-p/366514
try updating the software first,if nothing happens or no change
i would return it to a nokia care center if its original
If i have helped at all a click on the white star below would be nice thanks.
Now using the Lumia 1520

Blackberry Z10 "Call Failure"

Hi Can anyone help? My Blackberry Z10 has been working fine for a few months. This week I can not make calls nor can I take calls. When I dial out the phone displays "Call Failure" Contact your service provider. I have been in contact with Virgin and they say nothing is wrong with the account or sim. I have the sim in another blackberry phone and everything works just fine. I have done the security wipe, still the same fault. Does anyone have any ideas what I can do. I purchased the phone online "New" from a private seller.

I would get the sim replaced. I have seen similar issues with sim cards. I had a problem with my data plan on my sim. Wouldn't work on my Z10 but worked in a 9810. One think it was a problem with the device. Turned out that a replacement sim solved the problem.
Get a new sim.
1. Please thank those who help you by clicking the "Like" button at the bottom of the post that helped you.
2. If your issue has been solved, please resolve it by marking the post "Solution?" which solved it for you!

Failure 1 contacting cssd daemon

Similar Messages

Maybe you are looking for