Clusterware ASM & Listener OFFLINE

Hi guys..
When I tried to start the ASM & Listener on both nodes its successfully online after some times its goes OFFLINE, please any idea how to troubleshoot.
OS: Linux AS4
Clusterware: 10.2.0.1
On VMWARE machines..
./crs_stat -t
ora....SM1.asm application ONLINE OFFLINE
ora....C1.lsnr application ONLINE OFFLINE
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application ONLINE OFFLINE
ora....C2.lsnr application ONLINE OFFLINE
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac1
CRSD.log--->
2012-01-29 07:59:54.374: [    RACG][4143856384] [11769][4143856384][ora.rac1.vip]: Interface eth0 checked failed (host=rac1)
Invalid parameters, or failed to bring up VIP (host=rac1)
2012-01-29 07:59:54.375: [    RACG][4143856384] [11769][4143856384][ora.rac1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/crs
2012-01-29 07:59:54.375: [    RACG][4143856384] [11769][4143856384][ora.rac1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 54 /oracle/product/10.2.0/crs/bin/racgvip check rac1
2012-01-29 07:59:54.375: [    RACG][4143856384] [11769][4143856384][ora.rac1.vip]: clsrcexecut: rc = 1, time = 7.040s
2012-01-29 07:59:54.376: [    RACG][4143856384] [11769][4143856384][ora.rac1.vip]: end for resource = ora.rac1.vip, action = check, status = 1, time = 7.680s
2012-01-29 08:02:46.955: [    RACG][4143856384] [15062][4143856384][ora.rac1.vip]: Interface eth0 checked failed (host=rac1)
Invalid parameters, or failed to bring up VIP (host=rac1)
2012-01-29 08:02:46.956: [    RACG][4143856384] [15062][4143856384][ora.rac1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/crs
2012-01-29 08:02:46.956: [    RACG][4143856384] [15062][4143856384][ora.rac1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 54 /oracle/product/10.2.0/crs/bin/racgvip start rac1
2012-01-29 08:02:46.956: [    RACG][4143856384] [15062][4143856384][ora.rac1.vip]: clsrcexecut: rc = 1, time = 9.710s
2012-01-29 12:24:35.314: [    RACG][4143856384] [5546][4143856384][ora.rac1.vip]: Interface eth0 checked failed (host=rac1)
Invalid parameters, or failed to bring up VIP (host=rac1)
2012-01-29 12:24:35.337: [    RACG][4143856384] [5546][4143856384][ora.rac1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/crs
2012-01-29 12:24:35.337: [    RACG][4143856384] [5546][4143856384][ora.rac1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 54 /oracle/product/10.2.0/crs/bin/racgvip check rac1
2012-01-29 12:24:35.337: [    RACG][4143856384] [5546][4143856384][ora.rac1.vip]: clsrcexecut: rc = 1, time = 7.230s
2012-01-29 12:24:35.338: [    RACG][4143856384] [5546][4143856384][ora.rac1.vip]: end for resource = ora.rac1.vip, action = check, status = 1, time = 7.440s
2012-01-29 15:19:43.285: [    RACG][4143856384] [7534][4143856384][ora.rac1.vip]: Sun Jan 29 15:19:29 AST 2012 [ 7547 ] Broadcast = 192.168.1.255
Sun Jan 29 15:19:29 AST 2012 [ 7547 ] Checking interface existance
Sun Jan 29 15:19:29 AST 2012 [ 7547 ] Calling getifbyip
Sun Jan 29 15:19:29 AST 2012 [ 7547 ] getifbyip: started for 192.16
2012-01-29 15:19:43.489: [    RACG][4143856384] [7534][4143856384][ora.rac1.vip]: 8.1.101
Sun Jan 29 15:19:29 AST 2012 [ 7547 ] Completed getifbyip
Sun Jan 29 15:19:29 AST 2012 [ 7547 ] Calling getifbyip -a
Sun Jan 29 15:19:29 AST 2012 [ 7547 ] getifbyip: started for 192.168.1.101
Sun Jan 29 15:19:29 AST 2012 [ 7547 ] Completed getifb
2012-01-29 15:19:43.489: [    RACG][4143856384] [7534][4143856384][ora.rac1.vip]: yip
Sun Jan 29 15:19:32 AST 2012 [ 7547 ] Completed with initial interface test
Sun Jan 29 15:19:32 AST 2012 [ 7547 ] Interface tests
Sun Jan 29 15:19:32 AST 2012 [ 7547 ] checkIf: start for if=eth0
Sun Jan 29 15:19:32 AST 2012 [ 7547 ] /sbin/mii-tool eth
2012-01-29 15:19:43.490: [    RACG][4143856384] [7534][4143856384][ora.rac1.vip]: 0 error
Sun Jan 29 15:19:38 AST 2012 [ 7547 ] checkIf: RX packets checked if=eth0 OK
Sun Jan 29 15:19:38 AST 2012 [ 7547 ] checkIf: end for if=eth0
Sun Jan 29 15:19:39 AST 2012 [ 7547 ] getnextli: started for if=eth0
Sun Jan 29 15:19:39 AST 2012 [ 7547 ]
2012-01-29 15:19:43.490: [    RACG][4143856384] [7534][4143856384][ora.rac1.vip]: listif: starting
Sun Jan 29 15:19:39 AST 2012 [ 7547 ] listif: completed with eth0
eth1
Sun Jan 29 15:19:39 AST 2012 [ 7547 ] getnextli: completed with nextli=eth0:1
Sun Jan 29 15:19:39 AST 2012 [ 7547 ] Success exit 1
2012-01-29 15:44:23.453: [    RACG][4143856384] [4586][4143856384][ora.rac1.vip]: Sun Jan 29 15:44:16 AST 2012 [ 4590 ] Broadcast = 192.168.1.255
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] Checking interface existance
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] Calling getifbyip
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] getifbyip: started for 192.16
2012-01-29 15:44:23.454: [    RACG][4143856384] [4586][4143856384][ora.rac1.vip]: 8.1.101
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] getifbyip: returning IP eth0:1
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] Completed getifbyip eth0:1
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] Calling getifbyip -a
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] getifbyip: sta
2012-01-29 15:44:23.454: [    RACG][4143856384] [4586][4143856384][ora.rac1.vip]: rted for 192.168.1.101
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] getifbyip: returning IP eth0:1
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] Completed getifbyip eth0:1
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] Completed with initial interface test
Sun Jan 29 15:44:16 A
2012-01-29 15:44:23.454: [    RACG][4143856384] [4586][4143856384][ora.rac1.vip]: ST 2012 [ 4590 ] checkIf: start for if=eth0
Sun Jan 29 15:44:16 AST 2012 [ 4590 ] /sbin/mii-tool eth0 error
Sun Jan 29 15:44:23 AST 2012 [ 4590 ] checkIf: ping and RX packets checked if=eth0 failed
Interface eth0 checked failed (host=rac1)
Sun Jan 29 15:4
2012-01-29 15:44:23.454: [    RACG][4143856384] [4586][4143856384][ora.rac1.vip]: 4:23 AST 2012 [ 4590 ] checkIf: end for if=eth0
Sun Jan 29 15:44:23 AST 2012 [ 4590 ] Performing CRS_STAT testing
Sun Jan 29 15:44:23 AST 2012 [ 4590 ] Completed CRS_STAT testing
Sun Jan 29 15:44:23 AST 2012 [ 4590 ] Completed second gateway test
Sun Jan
2012-01-29 15:44:23.455: [    RACG][4143856384] [4586][4143856384][ora.rac1.vip]: 29 15:44:23 AST 2012 [ 4590 ] Interface tests
Invalid parameters, or failed to bring up VIP (host=rac1)
2012-01-29 15:44:23.455: [    RACG][4143856384] [4586][4143856384][ora.rac1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/crs
2012-01-29 15:44:23.455: [    RACG][4143856384] [4586][4143856384][ora.rac1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 54 /oracle/product/10.2.0/crs/bin/racgvip check rac1
2012-01-29 15:44:23.455: [    RACG][4143856384] [4586][4143856384][ora.rac1.vip]: clsrcexecut: rc = 1, time = 7.430s
2012-01-29 15:44:23.456: [    RACG][4143856384] [4586][4143856384][ora.rac1.vip]: end for resource = ora.rac1.vip, action = check, status = 1, time = 7.580s
2012-01-29 15:44:27.598: [    RACG][4143856384] [4790][4143856384][ora.rac1.vip]: Sun Jan 29 15:44:24 AST 2012 [ 4802 ] Broadcast = 192.168.1.255
Sun Jan 29 15:44:24 AST 2012 [ 4802 ] Checking interface existance
Sun Jan 29 15:44:24 AST 2012 [ 4802 ] Calling getifbyip
Sun Jan 29 15:44:24 AST 2012 [ 4802 ] getifbyip: started for 192.16
2012-01-29 15:44:27.599: [    RACG][4143856384] [4790][4143856384][ora.rac1.vip]: 8.1.101
Sun Jan 29 15:44:24 AST 2012 [ 4802 ] Completed getifbyip
Sun Jan 29 15:44:24 AST 2012 [ 4802 ] Calling getifbyip -a
Sun Jan 29 15:44:24 AST 2012 [ 4802 ] getifbyip: started for 192.168.1.101
Sun Jan 29 15:44:24 AST 2012 [ 4802 ] Completed getifb
2012-01-29 15:44:27.599: [    RACG][4143856384] [4790][4143856384][ora.rac1.vip]: yip
2012-01-30 07:21:22.240: [    RACG][4143856384] [22511][4143856384][ora.rac1.vip]: Mon Jan 30 07:21:09 AST 2012 [ 22515 ] Broadcast = 192.168.1.255
Mon Jan 30 07:21:09 AST 2012 [ 22515 ] Checking interface existance
Mon Jan 30 07:21:09 AST 2012 [ 22515 ] Calling getifbyip
Mon Jan 30 07:21:09 AST 2012 [ 22515 ] getifbyip: started for 19
2012-01-30 07:21:22.246: [    RACG][4143856384] [22511][4143856384][ora.rac1.vip]: 2.168.1.101
Mon Jan 30 07:21:09 AST 2012 [ 22515 ] Completed getifbyip
Mon Jan 30 07:21:09 AST 2012 [ 22515 ] Calling getifbyip -a
Mon Jan 30 07:21:09 AST 2012 [ 22515 ] getifbyip: started for 192.168.1.101
Mon Jan 30 07:21:09 AST 2012 [ 22515 ] Complete
2012-01-30 07:21:22.248: [    RACG][4143856384] [22511][4143856384][ora.rac1.vip]: d getifbyip
Mon Jan 30 07:21:12 AST 2012 [ 22515 ] Completed with initial interface test
Mon Jan 30 07:21:12 AST 2012 [ 22515 ] Interface tests
Mon Jan 30 07:21:12 AST 2012 [ 22515 ] checkIf: start for if=eth0
Mon Jan 30 07:21:12 AST 2012 [ 22515 ] /sbin/
2012-01-30 07:21:22.249: [    RACG][4143856384] [22511][4143856384][ora.rac1.vip]: mii-tool eth0 error
Mon Jan 30 07:21:18 AST 2012 [ 22515 ] checkIf: RX packets checked if=eth0 OK
Mon Jan 30 07:21:18 AST 2012 [ 22515 ] checkIf: end for if=eth0
Mon Jan 30 07:21:19 AST 2012 [ 22515 ] getnextli: started for if=eth0
Mon Jan 30 07:21:19 AS
2012-01-30 07:21:22.249: [    RACG][4143856384] [22511][4143856384][ora.rac1.vip]: T 2012 [ 22515 ] listif: starting
Mon Jan 30 07:21:19 AST 2012 [ 22515 ] listif: completed with eth0
eth1
Mon Jan 30 07:21:19 AST 2012 [ 22515 ] getnextli: completed with nextli=eth0:1
Mon Jan 30 07:21:19 AST 2012 [ 22515 ] Success exit 1
2012-01-30 07:23:38.416: [    RACG][4143856384] [24978][4143856384][ora.rac1.vip]: Mon Jan 30 07:23:31 AST 2012 [ 24987 ] Broadcast = 192.168.1.255
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] Checking interface existance
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] Calling getifbyip
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] getifbyip: started for 19
2012-01-30 07:23:38.420: [    RACG][4143856384] [24978][4143856384][ora.rac1.vip]: 2.168.1.101
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] getifbyip: returning IP eth0:1
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] Completed getifbyip eth0:1
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] Calling getifbyip -a
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] getifby
2012-01-30 07:23:38.420: [    RACG][4143856384] [24978][4143856384][ora.rac1.vip]: ip: started for 192.168.1.101
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] getifbyip: returning IP eth0:1
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] Completed getifbyip eth0:1
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] Completed with initial interface test
Mon Jan 30
2012-01-30 07:23:38.420: [    RACG][4143856384] [24978][4143856384][ora.rac1.vip]: 07:23:31 AST 2012 [ 24987 ] checkIf: start for if=eth0
Mon Jan 30 07:23:31 AST 2012 [ 24987 ] /sbin/mii-tool eth0 error
Mon Jan 30 07:23:38 AST 2012 [ 24987 ] checkIf: ping and RX packets checked if=eth0 failed
Interface eth0 checked failed (host=rac1)
M
2012-01-30 07:23:38.421: [    RACG][4143856384] [24978][4143856384][ora.rac1.vip]: on Jan 30 07:23:38 AST 2012 [ 24987 ] checkIf: end for if=eth0
Mon Jan 30 07:23:38 AST 2012 [ 24987 ] Performing CRS_STAT testing
Mon Jan 30 07:23:38 AST 2012 [ 24987 ] Completed CRS_STAT testing
Mon Jan 30 07:23:38 AST 2012 [ 24987 ] Completed second gat
2012-01-30 07:23:38.421: [    RACG][4143856384] [24978][4143856384][ora.rac1.vip]: eway test
Mon Jan 30 07:23:38 AST 2012 [ 24987 ] Interface tests
Invalid parameters, or failed to bring up VIP (host=rac1)
2012-01-30 07:23:38.421: [    RACG][4143856384] [24978][4143856384][ora.rac1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/crs
2012-01-30 07:23:38.422: [    RACG][4143856384] [24978][4143856384][ora.rac1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 54 /oracle/product/10.2.0/crs/bin/racgvip check rac1
2012-01-30 07:23:38.422: [    RACG][4143856384] [24978][4143856384][ora.rac1.vip]: clsrcexecut: rc = 1, time = 7.420s
2012-01-30 07:23:38.424: [    RACG][4143856384] [24978][4143856384][ora.rac1.vip]: end for resource = ora.rac1.vip, action = check, status = 1, time = 7.710s
2012-01-30 07:23:42.709: [    RACG][4143856384] [25171][4143856384][ora.rac1.vip]: Mon Jan 30 07:23:39 AST 2012 [ 25184 ] Broadcast = 192.168.1.255
Mon Jan 30 07:23:39 AST 2012 [ 25184 ] Checking interface existance
Mon Jan 30 07:23:39 AST 2012 [ 25184 ] Calling getifbyip
Mon Jan 30 07:23:39 AST 2012 [ 25184 ] getifbyip: started for 19
2012-01-30 07:23:42.711: [    RACG][4143856384] [25171][4143856384][ora.rac1.vip]: 2.168.1.101
Mon Jan 30 07:23:39 AST 2012 [ 25184 ] Completed getifbyip
Mon Jan 30 07:23:39 AST 2012 [ 25184 ] Calling getifbyip -a
Mon Jan 30 07:23:39 AST 2012 [ 25184 ] getifbyip: started for 192.168.1.101
Mon Jan 30 07:23:39 AST 2012 [ 25184 ] Complete
2012-01-30 07:23:42.711: [    RACG][4143856384] [25171][4143856384][ora.rac1.vip]: d getifbyip
vip2.log--->
Oracle Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 Oracle. All rights reserved.
2012-01-29 11:12:18.305: [    RACG][4143856384] [13419][4143856384][ora.rac2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/crs
2012-01-29 11:12:18.305: [    RACG][4143856384] [13419][4143856384][ora.rac2.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 54 /oracle/product/10.2.0/crs/bin/racgvip check rac2
2012-01-29 11:12:18.305: [    RACG][4143856384] [13419][4143856384][ora.rac2.vip]: clsrcexecut: rc = 1, time = 17.180s
2012-01-29 11:12:18.307: [    RACG][4143856384] [13419][4143856384][ora.rac2.vip]: end for resource = ora.rac2.vip, action = check, status = 1, time = 26.100s
2012-01-29 11:19:36.185: [    RACG][4143856384] [18998][4143856384][ora.rac2.vip]: Interface eth0 checked failed (host=rac1)
Invalid parameters, or failed to bring up VIP (host=rac1)
2012-01-29 11:19:36.220: [    RACG][4143856384] [18998][4143856384][ora.rac2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/crs
2012-01-29 11:19:36.220: [    RACG][4143856384] [18998][4143856384][ora.rac2.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 54 /oracle/product/10.2.0/crs/bin/racgvip check rac2
2012-01-29 11:19:36.220: [    RACG][4143856384] [18998][4143856384][ora.rac2.vip]: clsrcexecut: rc = 1, time = 7.150s
2012-01-29 11:19:36.220: [    RACG][4143856384] [18998][4143856384][ora.rac2.vip]: end for resource = ora.rac2.vip, action = check, status = 1, time = 7.310s
2012-01-29 14:51:02.108: [  OCRUTL][4143856384]u_freem: mem passed is null
2012-01-29 15:19:39.104: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: Sun Jan 29 15:19:30 AST 2012 [ 7553 ] Broadcast = 192.168.1.255
Sun Jan 29 15:19:30 AST 2012 [ 7553 ] Checking interface existance
Sun Jan 29 15:19:30 AST 2012 [ 7553 ] Calling getifbyip
Sun Jan 29 15:19:30 AST 2012 [ 7553 ] getifbyip: started for 192.16
2012-01-29 15:19:39.118: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: 8.1.201
Sun Jan 29 15:19:30 AST 2012 [ 7553 ] Completed getifbyip
Sun Jan 29 15:19:30 AST 2012 [ 7553 ] Calling getifbyip -a
Sun Jan 29 15:19:30 AST 2012 [ 7553 ] getifbyip: started for 192.168.1.201
Sun Jan 29 15:19:31 AST 2012 [ 7553 ] Completed getifb
2012-01-29 15:19:39.119: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: yip
Sun Jan 29 15:19:34 AST 2012 [ 7553 ] Completed with initial interface test
Sun Jan 29 15:19:34 AST 2012 [ 7553 ] Interface tests
Sun Jan 29 15:19:34 AST 2012 [ 7553 ] checkIf: start for if=eth0
Sun Jan 29 15:19:34 AST 2012 [ 7553 ] /sbin/mii-tool eth
2012-01-29 15:19:39.304: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: 0 error
Sun Jan 29 15:19:39 AST 2012 [ 7553 ] checkIf: ping and RX packets checked if=eth0 failed
Interface eth0 checked failed (host=rac1)
Sun Jan 29 15:19:39 AST 2012 [ 7553 ] checkIf: end for if=eth0
Invalid parameters, or failed to bring up VIP (host=
2012-01-29 15:19:39.304: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: rac1)
2012-01-29 15:19:39.304: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/crs
2012-01-29 15:19:39.304: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 54 /oracle/product/10.2.0/crs/bin/racgvip start rac2
2012-01-29 15:19:39.305: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: clsrcexecut: rc = 1, time = 10.180s
2012-01-29 15:19:52.320: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: Sun Jan 29 15:19:39 AST 2012 [ 7818 ] Broadcast = 192.168.1.255
Sun Jan 29 15:19:39 AST 2012 [ 7818 ] Checking interface existance
Sun Jan 29 15:19:39 AST 2012 [ 7818 ] Calling getifbyip
Sun Jan 29 15:19:39 AST 2012 [ 7818 ] getifbyip: started for 192.16
2012-01-29 15:19:52.320: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: 8.1.201
Sun Jan 29 15:19:39 AST 2012 [ 7818 ] Completed getifbyip
Sun Jan 29 15:19:39 AST 2012 [ 7818 ] Calling getifbyip -a
Sun Jan 29 15:19:40 AST 2012 [ 7818 ] getifbyip: started for 192.168.1.201
Sun Jan 29 15:19:40 AST 2012 [ 7818 ] Completed getifb
2012-01-29 15:19:52.320: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: yip
Sun Jan 29 15:19:43 AST 2012 [ 7818 ] Completed with initial interface test
Sun Jan 29 15:19:43 AST 2012 [ 7818 ] Performing CRS_STAT testing
Sun Jan 29 15:19:43 AST 2012 [ 7818 ] Completed CRS_STAT testing
Sun Jan 29 15:19:43 AST 2012 [ 7818 ] Interf
2012-01-29 15:19:52.320: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: ace tests
Sun Jan 29 15:19:43 AST 2012 [ 7818 ] checkIf: start for if=eth0
Sun Jan 29 15:19:43 AST 2012 [ 7818 ] /sbin/mii-tool eth0 error
Sun Jan 29 15:19:49 AST 2012 [ 7818 ] checkIf: RX packets checked if=eth0 OK
Sun Jan 29 15:19:49 AST 2012 [ 7818 ] c
2012-01-29 15:19:52.320: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: heckIf: end for if=eth0
Sun Jan 29 15:19:49 AST 2012 [ 7818 ] getnextli: started for if=eth0
Sun Jan 29 15:19:49 AST 2012 [ 7818 ] listif: starting
Sun Jan 29 15:19:49 AST 2012 [ 7818 ] listif: completed with eth0
eth0:1
eth1
Sun Jan 29 15:19:49 AST 201
2012-01-29 15:19:52.321: [    RACG][4143856384] [7532][4143856384][ora.rac2.vip]: 2 [ 7818 ] getnextli: completed with nextli=eth0:2
Sun Jan 29 15:19:49 AST 2012 [ 7818 ] Success exit 1
2012-01-29 15:19:52.582: [  OCRUTL][4143856384]u_freem: mem passed is null
2012-01-29 15:19:54.451: [    RACG][4143856384] [8119][4143856384][ora.rac2.vip]: Sun Jan 29 15:19:54 AST 2012 [ 8123 ] Broadcast = 192.168.1.255
Sun Jan 29 15:19:54 AST 2012 [ 8123 ] Checking interface existance
Sun Jan 29 15:19:54 AST 2012 [ 8123 ] Calling getifbyip
Sun Jan 29 15:19:54 AST 2012 [ 8123 ] getifbyip: started for 192.16
2012-01-29 15:19:54.454: [    RACG][4143856384] [8119][4143856384][ora.rac2.vip]: 8.1.201
Sun Jan 29 15:19:54 AST 2012 [ 8123 ] getifbyip: returning IP eth0:2
Sun Jan 29 15:19:54 AST 2012 [ 8123 ] Completed getifbyip eth0:2
Sun Jan 29 15:19:54 AST 2012 [ 8123 ] Calling getifbyip -a
Sun Jan 29 15:19:54 AST 2012 [ 8123 ] getifbyip: sta
2012-01-29 15:19:54.454: [    RACG][4143856384] [8119][4143856384][ora.rac2.vip]: rted for 192.168.1.201
Sun Jan 29 15:19:54 AST 2012 [ 8123 ] getifbyip: returning IP eth0:2
Sun Jan 29 15:19:54 AST 2012 [ 8123 ] Completed getifbyip eth0:2
Sun Jan 29 15:19:54 AST 2012 [ 8123 ] Completed with initial interface test
2012-01-29 15:46:39.596: [    RACG][4143856384] [7176][4143856384][ora.rac2.vip]: Sun Jan 29 15:46:26 AST 2012 [ 7194 ] Broadcast = 192.168.1.255
Sun Jan 29 15:46:26 AST 2012 [ 7194 ] Checking interface existance
Sun Jan 29 15:46:26 AST 2012 [ 7194 ] Calling getifbyip
Sun Jan 29 15:46:27 AST 2012 [ 7194 ] getifbyip: started for 192.16
2012-01-29 15:46:39.597: [    RACG][4143856384] [7176][4143856384][ora.rac2.vip]: 8.1.201
Sun Jan 29 15:46:27 AST 2012 [ 7194 ] Completed getifbyip
Sun Jan 29 15:46:27 AST 2012 [ 7194 ] Calling getifbyip -a
Sun Jan 29 15:46:27 AST 2012 [ 7194 ] getifbyip: started for 192.168.1.201
Sun Jan 29 15:46:27 AST 2012 [ 7194 ] Completed getifb
2012-01-29 15:46:39.597: [    RACG][4143856384] [7176][4143856384][ora.rac2.vip]: yip
Sun Jan 29 15:46:30 AST 2012 [ 7194 ] Completed with initial interface test
Sun Jan 29 15:46:30 AST 2012 [ 7194 ] Interface tests
Sun Jan 29 15:46:30 AST 2012 [ 7194 ] checkIf: start for if=eth0
Sun Jan 29 15:46:30 AST 2012 [ 7194 ] /sbin/mii-tool eth
2012-01-29 15:46:39.597: [    RACG][4143856384] [7176][4143856384][ora.rac2.vip]: 0 error
Sun Jan 29 15:46:36 AST 2012 [ 7194 ] checkIf: RX packets checked if=eth0 OK
Sun Jan 29 15:46:36 AST 2012 [ 7194 ] checkIf: end for if=eth0
Sun Jan 29 15:46:36 AST 2012 [ 7194 ] getnextli: started for if=eth0
Sun Jan 29 15:46:36 AST 2012 [ 7194 ]
2012-01-29 15:46:39.597: [    RACG][4143856384] [7176][4143856384][ora.rac2.vip]: listif: starting
Sun Jan 29 15:46:36 AST 2012 [ 7194 ] listif: completed with eth0
eth1
Sun Jan 29 15:46:36 AST 2012 [ 7194 ] getnextli: completed with nextli=eth0:1
Sun Jan 29 15:46:36 AST 2012 [ 7194 ] Success exit 1
2012-01-29 15:46:39.682: [  OCRUTL][4143856384]u_freem: mem passed is null
2012-01-30 07:20:44.752: [    RACG][4143856384] [21973][4143856384][ora.rac2.vip]: Mon Jan 30 07:20:44 AST 2012 [ 21980 ] Broadcast = 192.168.1.255
Mon Jan 30 07:20:44 AST 2012 [ 21980 ] Checking interface existance
Mon Jan 30 07:20:44 AST 2012 [ 21980 ] Calling getifbyip
Mon Jan 30 07:20:44 AST 2012 [ 21980 ] getifbyip: started for 19
2012-01-30 07:20:44.753: [    RACG][4143856384] [21973][4143856384][ora.rac2.vip]: 2.168.1.201
Mon Jan 30 07:20:44 AST 2012 [ 21980 ] getifbyip: returning IP eth0:1
Mon Jan 30 07:20:44 AST 2012 [ 21980 ] Completed getifbyip eth0:1
Mon Jan 30 07:20:44 AST 2012 [ 21980 ] Calling getifbyip -a
Mon Jan 30 07:20:44 AST 2012 [ 21980 ] getifby
2012-01-30 07:20:44.754: [    RACG][4143856384] [21973][4143856384][ora.rac2.vip]: ip: started for 192.168.1.201
Mon Jan 30 07:20:44 AST 2012 [ 21980 ] getifbyip: returning IP eth0:1
Mon Jan 30 07:20:44 AST 2012 [ 21980 ] Completed getifbyip eth0:1
Mon Jan 30 07:20:44 AST 2012 [ 21980 ] Completed with initial interface test

Hi,
Paste your listener.ora, /etc/hosts file contents from both nodes. It seem your vip on rac2 also not started. See if you can get the full stack started on at least one node.
Shutdown rac2 and restart rac1 see asm listener and nodeapps starts.

Similar Messages

  • ASM listener offline

    Following 455398.1 Using Oracle RAC and ASM with EBS 11i and 11g
    Finished step 3.6 Post-Oracle RAC Migration Steps
    Now ASM listener will not come ONLINE.
    When started manually, it terminates, no error message displayed.
    Questions:
    1- Is there a log file to gather more information as to why ASM listener dies?
    2- Some crs_stat command to get more info?
    3- Or is ASM not coming online, since we've not yet finished the note. The next step is to enable autoconfig on db tier.
    Rgds

    A
    lsnrctl servicesshould show the location of your logfile, like:
    Listener Log File /opt/asmprd/ora/diag/tnslsnr/vsv1h171ps/listener_vsv1h171ps/alert/log.xmlThis is a XML file which might be hard to read. Most of the time there is also a /opt/asmprd/ora/diag/tnslsnr/vsv1h171ps/listener_vsv1h171ps/trace directory, which contains a clear text logfile
    Ofcourse change the given directory name here to the ASM installation path of your installation

  • ASM, Listener, Instance status changes to OFFLINE.

    Hi All,
    I have completed the installation part...whereby installed Clusterware (no OCFS2), ASM and the RAC database S/w along with the database. The RAC version is Oracle 10.2.0.1. The installation is done on the CentOS operating system.
    The crs_stat -t shows all the resources. I am facing the following issue.
    1) After rebooting the server, most of the resources status changes to UNKNOWN. I need to stop all the resources and then need to restart it again (crs_start -all).
    2) The ASM, Listener and Instances resources status changes to OFFLINE after restarting all the resources. This happen regularly. Below is the output.
    [root@rac1 bin]# ./crs_stat -t
    Name Type Target State Host
    ora....B1.inst application ONLINE ONLINE rac1
    ora....B2.inst application ONLINE OFFLINE
    ora.RACDB.db application ONLINE ONLINE rac1
    ora....SM1.asm application ONLINE ONLINE rac1
    ora....C1.lsnr application ONLINE ONLINE rac1
    ora.rac1.gsd application ONLINE ONLINE rac1
    ora.rac1.ons application ONLINE ONLINE rac1
    ora.rac1.vip application ONLINE ONLINE rac1
    ora....SM2.asm application ONLINE OFFLINE
    ora....C2.lsnr application ONLINE OFFLINE
    ora.rac2.gsd application ONLINE ONLINE rac2
    ora.rac2.ons application ONLINE ONLINE rac2
    ora.rac2.vip application ONLINE ONLINE rac1
    If someone can help me with some solution for Point 2, it will be great.
    Regards
    Amit.

    Your Database started completely, right?
    You found ASM processes, right?
    You can check by:
    srvctl status asm -n <node_name>
    How?
    try to stop/start cluster again...
    Or try to start step by step -> nodeapps -> ASM -> database by use srvctl command-line
    anyway if you serious about resulting from crs_stat command-line:
    You need to unregister from Cluster and add it again.

  • Why the database, listener and asm are offline automatically

    I just install two nodes RAC on vmware, everything is OK. After I install database, I find, the system is not stable. I use the command
    "crs_stat -t" , everything is online, however, after a while, database instances, asms or listeners go often offline automatically.
    So, I have to use "srvctl start database -d" to start the database, after that, everything is online again.
    why is that? Anybody met the same situation before? Did I forget some important parameter?
    thanks

    check the crsd and cssd log file .
    also upgrade you clusterware,asm, db_home to 10.2.0.4.

  • Installing Clusterware/ASM Grid 11.2.0.1, errored out when running root.sh

    Working through the installation of Clusterware/ASM Grid using 11.2.0.1.
    After running the first root.sh script, no problems. On the second one, getting this error:
    Adding daemon to inittab
    CRS-4124: Oracle High Availability Services startup failed.
    CRS-4000: Command Start failed, or completed with errors.
    ohasd failed to start: Inappropriate ioctl for device
    ohasd failed to start: Inappropriate ioctl for device at /u01/app/grid/1120/grid/crs/install/roothas.pl line 413.
    Any ideas of how to get this running?

    Hi guys,
    I'm having this problem on the second node and in the /var/log files I'm getting these errors.
    Can any one help?
    Regards,
    Ahmed
    May 26 16:05:37 node su(pam_unix)[7567]: session closed for user oracle
    May 26 16:05:37 node logger: Failure in CSS initialization opening OCR.
    May 26 16:05:37 node su(pam_unix)[7571]: session closed for user oracle
    May 26 16:05:37 node logger: Failure in CSS initialization opening OCR.
    May 26 16:05:37 node su(pam_unix)[7570]: session closed for user oracle
    May 26 16:05:37 node logger: Failure in CSS initialization opening OCR.
    May 26 16:05:37 node su(pam_unix)[7583]: session closed for user oracle
    May 26 16:05:37 node logger: Failure in CSS initialization opening OCR.

  • Listener offline on node 2

    Dear all,
    When I give the command crs_stat -t
    Name Type Target State Host
    ora....S1.inst application ONLINE ONLINE db01
    ora....S2.inst application ONLINE ONLINE db02
    ora....TABS.db application ONLINE ONLINE db01
    ora....S1.lsnr application ONLINE ONLINE db01
    ora....b01.gsd application ONLINE ONLINE db01
    ora....b01.ons application ONLINE ONLINE db01
    ora....b01.vip application ONLINE ONLINE db01
    ora....S2.lsnr application ONLINE OFFLINE
    ora....b02.gsd application ONLINE ONLINE db02
    ora....b02.ons application ONLINE ONLINE db02
    ora....b02.vip application ONLINE ONLINE db01
    Shows listener offline... from the above, that the resource for the VIP is online at db01 only. ora....b02.vip application and ora....b01.vip application
    db01:oraccbs:/ccbs/appl/oracle$ srvctl start listener -n db02
    CRS-0210: Could not find resource ora.db02.LISTENER_db02.lsnr.
    Any idea how to resolve this ?
    which logfiles has to be verfied ?
    Please guide
    Kai

    Thanks Pal,
    I tried stopping i t
    db02:oraccbs:/ccbs/appl/oracle/product/102/network/log$ srvctl stop listener -n db02
    CRS-0210: Could not find resource ora.db02.LISTENER_DB02.lsnr.
    In the b/m location, I couldn't find any trace from the listener log file..where as imon_PROD.log shows the following :
    2008-10-31 09:26:52.962: [    RACG][21687] [6042][21687][ora.prod.db2.inst]: CLSR-0002: Oracle error encountered while executing clsrcsngtsn: fetch
    2008-10-31 09:26:52.962: [    RACG][21687] [6042][21687][ora.prod.db2.inst]: ORA-01405: fetched column value is NULL
    2008-10-31 09:36:53.621: [    RACG][21701] [6042][21701][ora.prod.db2.inst]: CLSR-0002: Oracle error encountered while executing clsrcsngtsn: fetch
    2008-10-31 09:36:53.621: [    RACG][21701] [6042][21701][ora.prod.db2.inst]: ORA-01405: fetched column value is NULL
    2008-10-31 09:46:54.181: [    RACG][21715] [6042][21715][ora.prod.db2.inst]: CLSR-0002: Oracle error encountered while executing clsrcsngtsn: fetch
    2008-10-31 09:46:54.181: [    RACG][21715] [6042][21715][ora.prod.db2.inst]: ORA-01405: fetched column value is NULL
    2008-10-31 09:56:54.741: [    RACG][21729] [6042][21729][ora.prod.db2.inst]: CLSR-0002: Oracle error encountered while executing clsrcsngtsn: fetch
    2008-10-31 09:56:54.741: [    RACG][21729] [6042][21729][ora.prod.db2.inst]: ORA-01405: fetched column value is NULL
    2008-10-31 10:06:55.292: [    RACG][21743] [6042][21743][ora.prod.db2.inst]: CLSR-0002: Oracle error encountered while executing clsrcsngtsn: fetch
    2008-10-31 10:06:55.292: [    RACG][21743] [6042][21743][ora.prod.db2.inst]: ORA-01405: fetched column value is NULL
    2008-10-31 10:16:55.841: [    RACG][21757] [6042][21757][ora.prod.db2.inst]: CLSR-0002: Oracle error encountered while executing clsrcsngtsn: fetch
    2008-10-31 10:16:55.841: [    RACG][21757] [6042][21757][ora.prod.db2.inst]: ORA-01405: fetched column value is NULL
    2008-10-31 10:26:56.390: [    RACG][21771] [6042][21771][ora.prod.db2.inst]: CLSR-0002: Oracle error encountered while executing clsrcsngtsn: fetch
    2008-10-31 10:26:56.390: [    RACG][21771] [6042][21771][ora.prod.db2.inst]: ORA-01405: fetched column value is NULL
    any idea ?
    Kai
    2008-10-31 10:36:56.903: [    RACG][21785] [6042][21785][ora.prod.db2.inst]: CLSR-0002: Oracle error encountered while executing clsrcsngtsn: fetch
    2008-10-31 10:36:56.903: [    RACG][21785] [6042][21785][ora.prod.db2.inst]: ORA-01405: fetched column value is NULL

  • Listener & ASM going OFFLINE frequently

    Hi,
    I Install successfully RAC on 2-node clusterware, After I root both nodes its going UNKNOWN state then I tried to bring ONLINE with SRVCTL its successfully ONLINE but after some time nearly 10-minuts the ASM & LISTENERS going OFFLINE both Node. please any idea how to trouble shoot.
    2-node cluterware on VMWare.
    clusterware version: 10.2.0.1
    DB version:10.2.0.1
    os: linux as4
    Home:
    CRS: /oracle/product/10.2.0/crs
    ASM: /oracle/product/10.2.0/asm
    RDBMS: /oracle/product/10.2.0/rdbms
    after reboot the nodes see the below :
    [root@rac1 bin]# ./crs_stat -t
    Name Type Target State Host
    ora....SM1.asm application ONLINE ONLINE rac1
    ora....C1.lsnr application ONLINE UNKNOWN rac1
    ora.rac1.gsd application ONLINE UNKNOWN rac1
    ora.rac1.ons application ONLINE UNKNOWN rac1
    ora.rac1.vip application ONLINE OFFLINE
    ora....SM2.asm application ONLINE UNKNOWN rac2
    ora....C2.lsnr application ONLINE OFFLINE
    ora.rac2.gsd application ONLINE UNKNOWN rac2
    ora.rac2.ons application ONLINE UNKNOWN rac2
    ora.rac2.vip application ONLINE ONLINE rac2
    ora.racdb.db application ONLINE UNKNOWN rac2
    ora....b1.inst application ONLINE UNKNOWN rac1
    ora....b2.inst application ONLINE OFFLINE
    After bring Online manually with SRVCTL utility:
    ora....SM1.asm application ONLINE OFFLINE
    ora....C1.lsnr application ONLINE OFFLINE
    ora.rac1.gsd application ONLINE ONLINE rac1
    ora.rac1.ons application ONLINE ONLINE rac1
    ora.rac1.vip application ONLINE ONLINE rac2
    ora....SM2.asm application ONLINE OFFLINE
    ora....C2.lsnr application ONLINE OFFLINE
    ora.rac2.gsd application ONLINE ONLINE rac2
    ora.rac2.ons application ONLINE ONLINE rac2
    ora.rac2.vip application ONLINE ONLINE rac1
    ora.racdb.db application OFFLINE OFFLINE
    ora....b1.inst application OFFLINE OFFLINE
    ora....b2.inst application ONLINE OFFLINE

    ora.rac1.vip.log_:
    2012-02-07 10:56:23.702: [    RACG][4143856384] [11290][4143856384][ora.rac1.vip]: d getifbyip
    2012-02-07 13:53:59.725: [    RACG][4143856384] [23948][4143856384][ora.rac1.vip]: Tue Feb 7 13:53:46 AST 2012 [ 23962 ] Broadcast = 192.168.1.255
    Tue Feb 7 13:53:46 AST 2012 [ 23962 ] Checking interface existance
    Tue Feb 7 13:53:46 AST 2012 [ 23962 ] Calling getifbyip
    Tue Feb 7 13:53:46 AST 2012 [ 23962 ] getifbyip: started for 19
    2012-02-07 13:53:59.726: [    RACG][4143856384] [23948][4143856384][ora.rac1.vip]: 2.168.1.102
    Tue Feb 7 13:53:46 AST 2012 [ 23962 ] Completed getifbyip
    Tue Feb 7 13:53:46 AST 2012 [ 23962 ] Calling getifbyip -a
    Tue Feb 7 13:53:46 AST 2012 [ 23962 ] getifbyip: started for 192.168.1.102
    Tue Feb 7 13:53:47 AST 2012 [ 23962 ] Complete
    2012-02-07 13:53:59.726: [    RACG][4143856384] [23948][4143856384][ora.rac1.vip]: d getifbyip
    Tue Feb 7 13:53:50 AST 2012 [ 23962 ] Completed with initial interface test
    Tue Feb 7 13:53:50 AST 2012 [ 23962 ] Interface tests
    Tue Feb 7 13:53:50 AST 2012 [ 23962 ] checkIf: start for if=eth0
    Tue Feb 7 13:53:50 AST 2012 [ 23962 ] /sbin/
    2012-02-07 13:53:59.727: [    RACG][4143856384] [23948][4143856384][ora.rac1.vip]: mii-tool eth0 error
    Tue Feb 7 13:53:50 AST 2012 [ 23962 ] defaultgw: started
    Tue Feb 7 13:53:50 AST 2012 [ 23962 ] defaultgw: completed with 10.10.1.1
    Tue Feb 7 13:53:56 AST 2012 [ 23962 ] checkIf: RX packets checked if=eth0 OK
    Tue Feb 7 13:53:56 AS
    2012-02-07 13:53:59.727: [    RACG][4143856384] [23948][4143856384][ora.rac1.vip]: T 2012 [ 23962 ] checkIf: end for if=eth0
    Tue Feb 7 13:53:56 AST 2012 [ 23962 ] getnextli: started for if=eth0
    Tue Feb 7 13:53:56 AST 2012 [ 23962 ] listif: starting
    Tue Feb 7 13:53:56 AST 2012 [ 23962 ] listif: completed with eth0
    eth1
    Tue Feb 7 13
    2012-02-07 13:53:59.727: [    RACG][4143856384] [23948][4143856384][ora.rac1.vip]: :53:56 AST 2012 [ 23962 ] getnextli: completed with nextli=eth0:1
    Tue Feb 7 13:53:56 AST 2012 [ 23962 ] Success exit 1
    2012-02-07 14:01:51.130: [    RACG][4143856384] [1054][4143856384][ora.rac1.vip]: Tue Feb 7 14:01:46 AST 2012 [ 1061 ] Broadcast = 192.168.1.255
    Tue Feb 7 14:01:46 AST 2012 [ 1061 ] Checking interface existance
    Tue Feb 7 14:01:46 AST 2012 [ 1061 ] Calling getifbyip
    Tue Feb 7 14:01:46 AST 2012 [ 1061 ] getifbyip: started for 192.16
    2012-02-07 14:01:51.130: [    RACG][4143856384] [1054][4143856384][ora.rac1.vip]: 8.1.102
    Tue Feb 7 14:01:46 AST 2012 [ 1061 ] getifbyip: returning IP eth0:1
    Tue Feb 7 14:01:46 AST 2012 [ 1061 ] Completed getifbyip eth0:1
    Tue Feb 7 14:01:46 AST 2012 [ 1061 ] Calling getifbyip -a
    Tue Feb 7 14:01:46 AST 2012 [ 1061 ] getifbyip: sta
    2012-02-07 14:01:51.131: [    RACG][4143856384] [1054][4143856384][ora.rac1.vip]: rted for 192.168.1.102
    Tue Feb 7 14:01:46 AST 2012 [ 1061 ] getifbyip: returning IP eth0:1
    Tue Feb 7 14:01:46 AST 2012 [ 1061 ] Completed getifbyip eth0:1
    Tue Feb 7 14:01:47 AST 2012 [ 1061 ] Completed with initial interface test
    Tue Feb 7 14:01:47 A
    2012-02-07 14:01:51.131: [    RACG][4143856384] [1054][4143856384][ora.rac1.vip]: ST 2012 [ 1061 ] checkIf: start for if=eth0
    Tue Feb 7 14:01:47 AST 2012 [ 1061 ] /sbin/mii-tool eth0 error
    Tue Feb 7 14:01:47 AST 2012 [ 1061 ] defaultgw: started
    Tue Feb 7 14:01:47 AST 2012 [ 1061 ] defaultgw: completed with 10.10.1.1
    Tue Feb 7 14:
    2012-02-07 14:01:51.131: [    RACG][4143856384] [1054][4143856384][ora.rac1.vip]: 01:50 AST 2012 [ 1061 ] checkIf: ping and RX packets checked if=eth0 failed
    Interface eth0 checked failed (host=rac1)
    Tue Feb 7 14:01:50 AST 2012 [ 1061 ] checkIf: end for if=eth0
    Tue Feb 7 14:01:50 AST 2012 [ 1061 ] Performing CRS_STAT testing
    Tue Feb
    2012-02-07 14:01:51.131: [    RACG][4143856384] [1054][4143856384][ora.rac1.vip]: 7 14:01:50 AST 2012 [ 1061 ] Completed CRS_STAT testing
    Tue Feb 7 14:01:50 AST 2012 [ 1061 ] Completed second gateway test
    Tue Feb 7 14:01:50 AST 2012 [ 1061 ] Interface tests
    Invalid parameters, or failed to bring up VIP (host=rac1)
    2012-02-07 14:01:51.131: [    RACG][4143856384] [1054][4143856384][ora.rac1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/crs
    2012-02-07 14:01:51.131: [    RACG][4143856384] [1054][4143856384][ora.rac1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 54 /oracle/product/10.2.0/crs/bin/racgvip check rac1
    2012-02-07 14:01:51.131: [    RACG][4143856384] [1054][4143856384][ora.rac1.vip]: clsrcexecut: rc = 1, time = 4.650s
    2012-02-07 14:01:51.132: [    RACG][4143856384] [1054][4143856384][ora.rac1.vip]: end for resource = ora.rac1.vip, action = check, status = 1, time = 4.770s
    2012-02-07 14:01:55.436: [    RACG][4143856384] [1220][4143856384][ora.rac1.vip]: Tue Feb 7 14:01:51 AST 2012 [ 1232 ] Broadcast = 192.168.1.255
    Tue Feb 7 14:01:51 AST 2012 [ 1232 ] Checking interface existance
    Tue Feb 7 14:01:52 AST 2012 [ 1232 ] Calling getifbyip
    Tue Feb 7 14:01:52 AST 2012 [ 1232 ] getifbyip: started for 192.16
    2012-02-07 14:01:55.438: [    RACG][4143856384] [1220][4143856384][ora.rac1.vip]: 8.1.102
    Tue Feb 7 14:01:52 AST 2012 [ 1232 ] Completed getifbyip
    Tue Feb 7 14:01:52 AST 2012 [ 1232 ] Calling getifbyip -a
    Tue Feb 7 14:01:52 AST 2012 [ 1232 ] getifbyip: started for 192.168.1.102
    Tue Feb 7 14:01:52 AST 2012 [ 1232 ] Completed getifb
    2012-02-07 14:01:55.438: [    RACG][4143856384] [1220][4143856384][ora.rac1.vip]: yip
    2012-02-07 14:06:19.920: [    RACG][4143856384] [6233][4143856384][ora.rac1.vip]: Tue Feb 7 14:06:06 AST 2012 [ 6237 ] Broadcast = 192.168.1.255
    Tue Feb 7 14:06:07 AST 2012 [ 6237 ] Checking interface existance
    Tue Feb 7 14:06:07 AST 2012 [ 6237 ] Calling getifbyip
    Tue Feb 7 14:06:07 AST 2012 [ 6237 ] getifbyip: started for 192.16
    2012-02-07 14:06:19.922: [    RACG][4143856384] [6233][4143856384][ora.rac1.vip]: 8.1.102
    Tue Feb 7 14:06:07 AST 2012 [ 6237 ] Completed getifbyip
    Tue Feb 7 14:06:07 AST 2012 [ 6237 ] Calling getifbyip -a
    Tue Feb 7 14:06:07 AST 2012 [ 6237 ] getifbyip: started for 192.168.1.102
    Tue Feb 7 14:06:07 AST 2012 [ 6237 ] Completed getifb
    2012-02-07 14:06:19.922: [    RACG][4143856384] [6233][4143856384][ora.rac1.vip]: yip
    Tue Feb 7 14:06:10 AST 2012 [ 6237 ] Completed with initial interface test
    Tue Feb 7 14:06:10 AST 2012 [ 6237 ] Interface tests
    Tue Feb 7 14:06:10 AST 2012 [ 6237 ] checkIf: start for if=eth0
    Tue Feb 7 14:06:10 AST 2012 [ 6237 ] /sbin/mii-tool eth
    2012-02-07 14:06:19.922: [    RACG][4143856384] [6233][4143856384][ora.rac1.vip]: 0 error
    Tue Feb 7 14:06:10 AST 2012 [ 6237 ] defaultgw: started
    Tue Feb 7 14:06:10 AST 2012 [ 6237 ] defaultgw: completed with 10.10.1.1
    Tue Feb 7 14:06:16 AST 2012 [ 6237 ] checkIf: RX packets checked if=eth0 OK
    Tue Feb 7 14:06:16 AST 2012 [ 6237 ]
    2012-02-07 14:06:19.923: [    RACG][4143856384] [6233][4143856384][ora.rac1.vip]: checkIf: end for if=eth0
    Tue Feb 7 14:06:16 AST 2012 [ 6237 ] getnextli: started for if=eth0
    Tue Feb 7 14:06:16 AST 2012 [ 6237 ] listif: starting
    Tue Feb 7 14:06:16 AST 2012 [ 6237 ] listif: completed with eth0
    eth0:2
    eth1
    Tue Feb 7 14:06:16 AST 2
    2012-02-07 14:06:19.923: [    RACG][4143856384] [6233][4143856384][ora.rac1.vip]: 012 [ 6237 ] getnextli: completed with nextli=eth0:1
    Tue Feb 7 14:06:16 AST 2012 [ 6237 ] Success exit 1
    2012-02-07 14:48:24.219: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]: Tue Feb 7 14:48:17 AST 2012 [ 21896 ] Broadcast = 192.168.1.255
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] Checking interface existance
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] Calling getifbyip
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] getifbyip: started for 19
    2012-02-07 14:48:24.220: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]: 2.168.1.102
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] getifbyip: returning IP eth0:1
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] Completed getifbyip eth0:1
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] Calling getifbyip -a
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] getifby
    2012-02-07 14:48:24.220: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]: ip: started for 192.168.1.102
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] getifbyip: returning IP eth0:1
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] Completed getifbyip eth0:1
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] Completed with initial interface test
    Tue Feb 7
    2012-02-07 14:48:24.220: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]: 14:48:17 AST 2012 [ 21896 ] checkIf: start for if=eth0
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] /sbin/mii-tool eth0 error
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] defaultgw: started
    Tue Feb 7 14:48:17 AST 2012 [ 21896 ] defaultgw: completed with 10.10.1.1
    2012-02-07 14:48:24.220: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]:
    Tue Feb 7 14:48:23 AST 2012 [ 21896 ] checkIf: ping and RX packets checked if=eth0 failed
    Interface eth0 checked failed (host=rac1)
    Tue Feb 7 14:48:23 AST 2012 [ 21896 ] checkIf: end for if=eth0
    Tue Feb 7 14:48:23 AST 2012 [ 21896 ] Performing CRS_STA
    2012-02-07 14:48:24.220: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]: T testing
    Tue Feb 7 14:48:24 AST 2012 [ 21896 ] Completed CRS_STAT testing
    Tue Feb 7 14:48:24 AST 2012 [ 21896 ] Completed second gateway test
    Tue Feb 7 14:48:24 AST 2012 [ 21896 ] Interface tests
    Invalid parameters, or failed to bring up VIP (host=rac
    2012-02-07 14:48:24.220: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]: 1)
    2012-02-07 14:48:24.220: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/crs
    2012-02-07 14:48:24.220: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/crs/bin/racgeut -e USRORA_DEBUG=0 54 /oracle/product/10.2.0/crs/bin/racgvip check rac1
    2012-02-07 14:48:24.220: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]: clsrcexecut: rc = 1, time = 7.230s
    2012-02-07 14:48:24.221: [    RACG][4143856384] [21892][4143856384][ora.rac1.vip]: end for resource = ora.rac1.vip, action = check, status = 1, time = 7.370s
    2012-02-07 14:48:28.525: [    RACG][4143856384] [22100][4143856384][ora.rac1.vip]: Tue Feb 7 14:48:25 AST 2012 [ 22110 ] Broadcast = 192.168.1.255
    Tue Feb 7 14:48:25 AST 2012 [ 22110 ] Checking interface existance
    Tue Feb 7 14:48:25 AST 2012 [ 22110 ] Calling getifbyip
    Tue Feb 7 14:48:25 AST 2012 [ 22110 ] getifbyip: started for 19
    2012-02-07 14:48:28.527: [    RACG][4143856384] [22100][4143856384][ora.rac1.vip]: 2.168.1.102
    Tue Feb 7 14:48:25 AST 2012 [ 22110 ] Completed getifbyip
    Tue Feb 7 14:48:25 AST 2012 [ 22110 ] Calling getifbyip -a
    Tue Feb 7 14:48:25 AST 2012 [ 22110 ] getifbyip: started for 192.168.1.102
    Tue Feb 7 14:48:25 AST 2012 [ 22110 ] Complete
    2012-02-07 14:48:28.527: [    RACG][4143856384] [22100][4143856384][ora.rac1.vip]: d getifbyip

  • A non-RAC question for the experts: Clusterware + ASM

    Hi
    We have a 2-nodes cluster in production.
    It's an active/passive cluster with a cold failover using SUN Cluster.
    There is only one Oracle 10G single-instance database (DB1), running on node #1, that use ASM to access the SAN (it's raw devices).
    We want to add a third nodes to move to a 3-nodes cluster, that would have 2 separates single instance Oracle 10g database
    node #1: DB1 with ASM (the existing database, existing raw devices)
    node #2: cluster passive node
    node #3: DB2 with ASM (new additional database, new raw devices)
    All nodes are connected to the same SAN.
    We' want to use Clusterware instead of SUN Cluster.
    The single-instance databases, the listeners and the ASM instances would be protected by Clusterware using "application VIP" (not oracle-vip).
    It's not RAC ! but in the future both database might become RAC (but that's not the need today)...
    My concern is about ASM !
    Do I have to install an ASM instance on each node (just like in RAC) ?
    Or may I just configure clusterware to ensure there is only ASM instance running on a node ?
    Is it possible to have the ASM instance of node #1 and the ASM intance of node#2 that access the same SAN & raw devices ?
    In the case DB1 & DB2 fail over to node#2, the unique ASM instance must see the raw devices of both databases, is this possible ?
    Do I have to destroy the curent ASM configuration of DB1 to build the new one, or can I just add new raw devices and share tem across all nodes ?
    Thanks for your answers.

    Hi,
    Thanks for the answer
    I found this very interesting doucument but it does not go into enough details about ASM, which is my concern.
    I found in the Oracle 11g documentation a statement which say that clustered ASM instances for single instance database do not need to talk each other as long as they do not handle the same diskgroup (Figure 1.3 ; http://download.oracle.com/docs/cd/B28359_01/server.111/b31107/asmcon.htm#i1021337) . Otherwise, it must be clustered. It would also be a problem for me when DB1 and DB2 are on the same node. I would need two different ASM instance, on the same node, which is not a good situation.
    What happen when I start an ASM instance on node #3 to access raw devices already in use by node # 2.
    I think I need to start from node #1 and change some ASM configuratoion to make it cluster-aware.
    Next, I would add a second clustered ASM instance on node #2, and so on, to make sure that all ASM instances of all nodes can access the same diskgroup, and thefore being able to re-use the existing database on node #1 as well as creating a new one one node #2 (all on the same SAN)

  • ASM listener

    i do the installation of asm then create the databas using DBCA on windows 2003 10gR2 on VMWare
    now i already check the docs but didn't find any solution to this problem.
    Can somebody tell me how to do this.
    lsnrctl status is as follows
    LSNRCTL> status
    Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.80.0.75)(PORT=1521)))
    STATUS of the LISTENER
    Alias LISTENER
    Version TNSLSNR for 32-bit Windows: Version 10.2.0.1.0 - Produ
    ction
    Start Date 17-SEP-2009 23:34:33
    Uptime 0 days 0 hr. 14 min. 15 sec
    Trace Level off
    Security ON: Local OS Authentication
    SNMP OFF
    Listener Parameter File C:\homedbse\oracle\product\10.2.0\db_1\network\admin\l
    istener.ora
    Listener Log File C:\homedbse\oracle\product\10.2.0\db_1\network\log\lis
    tener.log
    Listening Endpoints Summary...
    (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.80.0.75)(PORT=1521)))
    (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=\\.\pipe\EXTPROC0ipc)))
    Services Summary...
    Service "+ASM" has 2 instance(s).
    Instance "+ASM", status UNKNOWN, has 1 handler(s) for this service...
    Instance "+asm", status BLOCKED, has 1 handler(s) for this service...
    Service "+ASM_XPT" has 1 instance(s).
    Instance "+asm", status BLOCKED, has 1 handler(s) for this service...
    Service "ORAASM" has 2 instance(s).
    Instance "ORAASM", status UNKNOWN, has 1 handler(s) for this service...
    Instance "oraasm", status READY, has 1 handler(s) for this service...
    Service "PLSExtProc" has 1 instance(s).
    Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
    Service "oraasmXDB" has 1 instance(s).
    Instance "oraasm", status READY, has 1 handler(s) for this service...
    Service "oraasm_XPT" has 1 instance(s).
    Instance "oraasm", status READY, has 1 handler(s) for this service...
    The command completed successfully
    thanks

    I still don't get your problem. ASM is not a database instance,it never mounts a database, so status 'BLOCKED' is absolutely correct. And as long as you don't want to connect remotely to ASM,this is not an issue. There's nothing wrong in the 'lsnrctl status' output.
    then i have to start the listener manually everytime You have to start the listener manually,when startup modeof the service is MANUAL.
    Werner

  • ASM Disk offline - State hung

    Hey in my ASM configuration I setup this
    Disk Path ASM Name Failure Group
    /dev/raw/raw4 Data1 FG1_Data [on SAN1]
    /dev/raw/raw5 Data2 FG2_Data [on SAN2]
    /dev/raw/raw6 Reco1 FG1_Reco [on SAN1]
    /dev/raw/raw7 Reco2 FG2_Reco [on SAN2]
    all good.
    My I switched off SAN1. A query on v$asm_disk shows disk DATA1 & RECO1 as offline - hung state.
    I did in the asm instance in sqlplus:
    ALTER DISKGROUP DATA DROP DISK data1; -> success
    ALTER DISKGROUP RECO DROP DISK reco1; -> success
    ALTER DISKGROUP DATA ADD failgroup DATA3 '/dev/raw/raw4'; -> success
    ALTER DISKGROUP RECO ADD failgroup RECO3 '/dev/raw/raw6'; -> success
    The query on v$asm_disk stills show the DATA1 and RECO1 - but DATA3 and RECO3 as well.
    Is it right, that the "old" entries will disappear, once the ASM instance is restarted ? I can´t do this at the moment.
    In generell - why went the ASM disks to offline ?
    SAN1 was switched off for about 5 minutes - and was then switched back on.
    Is this expected behaviour ?

    Hi Chris,
    as long as the information is still partially valid, the devices will be displayed in v$asm_disk. Main reason is: Some Oracle processes still have handles open on these devices, and as long as they exist the devices can still be seen.
    A Restart of ASM + database will definitely remove these entries (if they cannot be seen by the operating system).
    Regarding your other question: Oracle can only detect if a device is going offline (errors in the database and asm alter log). If a device goes online again, Oracle does not check this (this would be a major overhead to rescan lets say every minute to see if a disk is back online).
    Hence you have to reinclude the dropped/offlined disks yourself.
    Depending on the version (10g or 11g) and depending on the diskgroup compatibility level for rdbms a disk can simply be returned into the diskgroup (11g - alter diskgroup <dg> set disk <disk> online) and resilvered or have to be reincluded into the ASM instance (no resilvering, but a complete rebalance 10g: alter diskgroup <dg> add disk <disk> to failgroup <fg> [force])
    Sebastian

  • Applying 11gR2 GI PSU with 10gR2 databases

    On a Linux-x86-64 cluster I have 11.2.0.1 Clusterware/ASM/Listener running without patches and a RAC database running under the old 10.2.0.4.1 home (PSU 1). I'm thinking of applying the GI PSU 9655006 for 11.2.0.1.2 but I notice it includes the database PSU for 11.2.0.1.2 and I am afraid the patch will attempt to update the 10.2.0.4 home.
    Looking at the Readme for the GI PSU I can't find any warnings about applying the patch with 10g databases present. Has anyone tried this? Thanks in advance.

    Hi Chucky,
    actually it will not.
    Before a patch is installed OPatch checks the inventory if the prerequisites are o.k. and since 10.2 is not a valid prerequisite for an 11.2 patch, it won't install into the DB home.
    The reason the patch includes the DB patch is, that the 11.2 GI home includes oracle db executable (for ASM) and that needs to be patched.
    However if you still fear that something may go wrong, you can explicitly tell opatch auto to only patch the GI home.
    It's in the readme or if you execute opatch auto -h:
    -oh Database/Clusterware home locations
    comma_delimited_list_of_oralcehomes_to_patch
    The default is all applicable Oracle Homes including
    the clusterware home .Use this option to patch RDBMS homes where
    no database is registered or any specific database home or
    clusterware home.
    Sebastian

  • Asm, instance and service failing to start 10g rac redhat el5

    hi,
    i have been following a how to guide from oracle's website about implementing a 10g rac system using el5 (http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_iscsi.html), by the end of the installation and setup I had both the nodes up and running and all services were up and running. Then under the root account I rebooted one of the nodes. now when I check the status of the services I get this:
    [oracle@rac01 ~]$ crs_stat -t
    Name           Type           Target    State     Host
    ora.orcl.db    application    ONLINE    ONLINE    rac01
    ora...._taf.cs application    ONLINE    ONLINE    rac02
    ora....cl1.srv application    ONLINE    OFFLINE
    ora....cl2.srv application    ONLINE    ONLINE    rac02
    ora....l1.inst application    OFFLINE   OFFLINE
    ora....l2.inst application    ONLINE    ONLINE    rac02
    ora....SM1.asm application    OFFLINE   OFFLINE
    ora....01.lsnr application    ONLINE    ONLINE    rac01
    ora.rac01.gsd  application    ONLINE    ONLINE    rac01
    ora.rac01.ons  application    ONLINE    ONLINE    rac01
    ora.rac01.vip  application    ONLINE    ONLINE    rac01
    ora....SM2.asm application    ONLINE    ONLINE    rac02
    ora....02.lsnr application    ONLINE    ONLINE    rac02
    ora.rac02.gsd  application    ONLINE    ONLINE    rac02
    ora.rac02.ons  application    ONLINE    ONLINE    rac02
    ora.rac02.vip  application    ONLINE    ONLINE    rac02though I can start the asm instance manually by
    srvctl start asm -n rac01and it starts but the instance and service on the rebooted node still wont start. is there a log file which tells me what the error is when it tries to start the instance? what can I do to ensure that if a node is rebooted that all of the services get started correctly and the node automatically rejoins the cluster (i thought I had set all of this from the instructions of the guide I was following).
    Regards
    Martin.

    Check ASM resource:
    crs_stat -p "resource_name
    $ crs_stat | grep NAME\= | grep ASM1
    NAME=ora.rac01.ASM1.asm
    $ crs_stat -p ora.rac01.ASM1.asm
    NAME=ora.rac01.ASM1.asm
    TYPE=application
    ACTIVE_PLACEMENT=0
    *AUTO_START*=1
    CHECK_INTERVAL=600
    DESCRIPTION=CRS application for ASM instance
    What do you see AUTO_START?
    if 1 = auto start:
    or check error ASM while start... at ASM_HOME/log/rac01/racg/*
    If AUTO_START=2
    Oracle Clusterware is configured to start the VIP, listener, instance, ASM, database services and other resources during system boot. Starting with 10.1.0.4, it is possible to modify some resources to have their profile parameter AUTO_START set to the value 2. This means that after node reboot, or when Oracle Clusterware is started, resources with AUTO_START=2 will need to be started manually via srvctl. Please work with Oracle Support if you need to modify this attribute.
    If AUTO_START=2 and need to change to 1:
    Example:
    $ crs_stat -p ora.rac01.ASM1.asm
    NAME=ora.rac01.ASM1.asm
    TYPE=application
    ACTIVE_PLACEMENT=0
    AUTO_START=2
    $ crs_stat -p ora.rac01.ASM1.asm > ORA_CRS_HOME/crs/public/ora.rac01.ASM1.asm
    $ crs_unregister ora.rac01.ASM1.asm
    $ crs_stat -p ora.rac01.ASM1.asm
    CRS-0210: Could not find resource 'ora.rac01.ASM1.asm
    $ vi ORA_CRS_HOME/crs/public/ora.rac01.ASM1.asm.cap
    AUTO_START=2
    to
    AUTO_START=1
    $ crs_register ora.rac01.ASM1.asm
    $ crs_stat -p ora.rac01.ASM1.asm
    NAME=ora.rac01.ASM1.asm
    TYPE=application
    *AUTO_START=1*
    Edited by: Surachart Opun (HunterX) on Jul 31, 2009 1:06 PM

  • ASM resource online and ASM idle instance

    Hi all,
    SO..........: CentOS 5.8 32bits (VM - VirtualBox)
    Oracle...: 11gR2 (11.2.0.1.0)
    On my test environment, i have a VM running Oracle Clusterware and 2 11g instances. The problem is: The ASM service appears online (through ./crsctl status resource -t, see below), but when a try to connect to the ASM instance as sysasm, it shows me that the instance is idle, and if i try to startup it, the message "ORA-15149: another ASM instance found running on the host" is shown to me.
    --output of ./crsctl status resource -t
    NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
    Local Resources
    ora.DATA.dg
                   ONLINE  ONLINE       vm-cent5                                    
    ora.DG_MANOBRA.dg
                   ONLINE  ONLINE       vm-cent5                                    
    ora.LISTENER.lsnr
                   ONLINE  ONLINE       vm-cent5                                    
    ora.asm
                   ONLINE  OFFLINE      vm-cent5                 Instance Shutdown  
    Cluster Resources
    ora.auxdb.db
          1        ONLINE  ONLINE       vm-cent5                 Open               
    ora.cssd
          1        ONLINE  ONLINE       vm-cent5                                    
    ora.diskmon
          1        ONLINE  ONLINE       vm-cent5 If i try to start the ora.asm resource through ./crtctl start, the following output is shown:
    [root@VM-Cent5 bin]# ./crsctl start resource ora.asm
    CRS-2672: Attempting to start 'ora.asm' on 'vm-cent5'
    ORA-01012: not logged on
    CRS-2674: Start of 'ora.asm' on 'vm-cent5' failed
    CRS-2679: Attempting to clean 'ora.asm' on 'vm-cent5'
    CRS-2681: Clean of 'ora.asm' on 'vm-cent5' succeeded
    CRS-4000: Command Start failed, or completed with errors.I am probably doing something stupid, but i still could not see what i'm doing wrong.
    Any help will be appreciated. Thanks in advance.

    Just one thing that i saw in my recent tests. I restarted CRS (crsctl stop | start resource -all), and the asm service (ora.asm) was back online, both target and state, as you can see below:
    [root@VM-Cent5 bin]# ./crsctl status resource -t
    NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
    Local Resources
    ora.DATA.dg
                   ONLINE  ONLINE       vm-cent5                                    
    ora.DG_MANOBRA.dg
                   ONLINE  ONLINE       vm-cent5                                    
    ora.LISTENER.lsnr
                   ONLINE  ONLINE       vm-cent5                                    
    ora.asm
                   ONLINE  ONLINE       vm-cent5                 Started            
    Cluster Resources
    ora.auxdb.db
          1        ONLINE  ONLINE       vm-cent5                 Open               
    ora.cssd
          1        ONLINE  ONLINE       vm-cent5                                    
    ora.diskmon
          1        ONLINE  ONLINE       vm-cent5                       ... but when i connect as sysasm using user oracle (sid +ASM), i see again the idle instance.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       

  • How to remove a node apps listener from CRS.

    We have a node apps in type of listener offline shown from crs_stat -t:
    ldctrdb01|PDAMLDV11|/u01/crs/product/10.2.0/bin $ crs_stat -t
    Name Type Target State Host
    ora....11.inst application ONLINE ONLINE ldctrdb01
    ora....12.inst application ONLINE ONLINE ldctrdb02
    ora....LDV1.db application ONLINE ONLINE ldctrdb02
    ora....SM1.asm application ONLINE ONLINE ldctrdb01
    ora....01.lsnr application ONLINE OFFLINE
    ora....01.lsnr application ONLINE ONLINE ldctrdb01
    ora....b01.gsd application ONLINE ONLINE ldctrdb01
    ora....b01.ons application ONLINE ONLINE ldctrdb01
    ora....b01.vip application ONLINE ONLINE ldctrdb01
    ora....SM2.asm application ONLINE ONLINE ldctrdb02
    ora....02.lsnr application ONLINE OFFLINE
    ora....02.lsnr application ONLINE ONLINE ldctrdb02
    ora....b02.gsd application ONLINE ONLINE ldctrdb02
    ora....b02.ons application ONLINE ONLINE ldctrdb02
    ora....b02.vip application ONLINE ONLINE ldctrdb02
    I want to delete it from the list by using netca. But this listener would not be able to show from netca. The netca can only show the other listener that is currently used and also shown in the crs_stat. How to remove the offline listener from crs_stat list?
    Thanks for advice!

    I run the netca from both $ORACLE_HOME and $ASM_HOME earlier. But now, none of the home can detect the listener to be deleted. However, if I try to add a listener with that name, the netca replied with " the listener already exist...".
    We have a symbolic link from $ORACLE_HOME/network/admin to $ASM_HOME/network/admin ($TNS_ADMIN) for the files tnsnames.ora and listener.ora.

  • 11g R2 Grid Infrastructure asmca cannot detect 11g R1 ASM instance

    I am playing with 11g ASM with XEN (one node on host (or dom0) and another node on vm (or dom1)) and was trying to do a Clusterware + ASM upgrade from 11.0.6 to 11.2.0. After solving some stupid problem, (see [rootupgrade.sh failed on 11g R1 to R2 Upgrade |http://forums.oracle.com/forums/thread.jspa?threadID=1005424&tstart=0]) so that all services can be started in both nodes, I am trying to upgrade ASM.
    I am trying asmca. But it is unable to detect the 11g R1 ASM instance. When started in GUI mode, it just offer to create ASM. When started in silent mode for upgrade, it shows:
    oracle@penguin-dom0:~> asmca -silent -upgradeASM
    Enter ASMSNMP Password
    ORA-01034: ORACLE not availableAnd I can't find anything in ASM alert log
    ASM is running from 11g R1 home
    oracle@penguin-dom0:~> ps -ef|grep asm
    oracle   21811     1  0 Jan06 ?        00:00:00 asm_pmon_+ASM1
    oracle   21813     1  0 Jan06 ?        00:00:03 asm_vktm_+ASM1
    oracle   21817     1  0 Jan06 ?        00:00:02 asm_diag_+ASM1
    oracle   21819     1  0 Jan06 ?        00:00:00 asm_ping_+ASM1
    oracle   21821     1  0 Jan06 ?        00:00:00 asm_psp0_+ASM1
    oracle   21825     1  0 Jan06 ?        00:00:09 asm_dia0_+ASM1
    oracle   21827     1  0 Jan06 ?        00:00:05 asm_lmon_+ASM1
    oracle   21829     1  0 Jan06 ?        00:00:03 asm_lmd0_+ASM1
    oracle   21831     1  0 Jan06 ?        00:00:03 asm_lms0_+ASM1
    oracle   21835     1  0 Jan06 ?        00:00:00 asm_mman_+ASM1
    oracle   21837     1  0 Jan06 ?        00:00:00 asm_dbw0_+ASM1
    oracle   21839     1  0 Jan06 ?        00:00:00 asm_lgwr_+ASM1
    oracle   21841     1  0 Jan06 ?        00:00:00 asm_ckpt_+ASM1
    oracle   21843     1  0 Jan06 ?        00:00:00 asm_smon_+ASM1
    oracle   21845     1  0 Jan06 ?        00:00:01 asm_rbal_+ASM1
    oracle   21847     1  0 Jan06 ?        00:00:00 asm_gmon_+ASM1
    oracle   21856     1  0 Jan06 ?        00:00:00 asm_lck0_+ASM1
    oracle   22021     1  0 Jan06 ?        00:00:00 /u01/app/oradb/product/11.1.0/db_1/bin/racgimon daemon ora.penguin-dom0.ASM1.asm11g R1 Listener is serving to ASM
    oracle@penguin-dom0:~> lsnrctl status
    LSNRCTL for Linux: Version 11.1.0.6.0 - Production on 07-JAN-2010 00:33:44
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
    STATUS of the LISTENER
    Alias                     LISTENER_PENGUIN-DOM0
    Version                   TNSLSNR for Linux: Version 11.1.0.6.0 - Production
    Start Date                06-JAN-2010 22:29:51
    Uptime                    0 days 2 hr. 3 min. 53 sec
    Trace Level               off
    Security                  ON: Local OS Authentication
    SNMP                      OFF
    Listener Parameter File   /u01/app/oradb/product/11.1.0/db_1/network/admin/listener.ora
    Listener Log File         /u01/app/oradb/diag/tnslsnr/penguin-dom0/listener_penguin-dom0/alert/log.xml
    Listening Endpoints Summary...
      (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.2.201)(PORT=1521)))
      (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.2.101)(PORT=1521)))
    Services Summary...
    Service "+ASM" has 1 instance(s).
      Instance "+ASM1", status READY, has 1 handler(s) for this service...
    Service "+ASM_XPT" has 1 instance(s).
      Instance "+ASM1", status READY, has 1 handler(s) for this service...
    The command completed successfullyNo tnsnames.ora in 11g R1 or Grid Infrastructure home.
    Setting ORACLE_HOME to 11g R1 home and ORACLE_SID=+ASM1, I can connect to ASM instance as sysasm using R1 sqlplus
    oracle@penguin-dom0:~> export ORACLE_HOME=/u01/app/oradb/product/11.1.0/db_1
    oracle@penguin-dom0:~> export ORACLE_SID=+ASM1
    oracle@penguin-dom0:~> $ORACLE_HOME/bin/sqlplus / as sysasm
    SQL*Plus: Release 11.1.0.6.0 - Production on Thu Jan 7 00:40:15 2010
    Copyright (c) 1982, 2007, Oracle.  All rights reserved.
    Connected to:
    Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit Production
    With the Partitioning, Real Application Clusters, Oracle Label Security, OLAP,
    Data Mining and Real Application Testing options
    SQL> Tried different combinations of ORACLE_HOME (11g R1 db home, grid infrastructure home, 11g R1 clusterware home) and ORACLE_SID (+ASM, +ASM1), but asmca still cannot connect to ASM instance.
    Some documents mentioned to configure 'Oracle Restart' and the ASM instance name is +ASM. Are these necessary requirements? If so, where can I find information on Oracle Restart and how to rename the ASM instance? Besides, the documents said it is recommanded to upgrade with OUI, but I've already installed the software, can I still use OUI?
    OS: OpenSUSE 11.1, XEN DOM0 and DOM1
    Oracle: 11.0.6 upgrade to 11.2.0
    Edited by: thtsang on Jan 6, 2010 8:47 AM
    Edited by: thtsang on Jan 6, 2010 5:33 PM

    Hi Thtsang,
    sorry for not coming back to you earlier and thanks for "the helpful".
    So, are you saying that after you have removed all the environment variables, you still got the error message "ORA-01034: ORACLE not available"? I thought you got an "ASM upgrade failed", because ASM was down at the time of the upgrade, which you then corrected saying: "In fact, ASM instance was started from Grid Infrastructure home.". Overall, I need to say that I am a bit confused to be honest.
    The alert log entry sounds promising, although, one would need to see more context. For: "But when I use crsctl / srvctl to start ASM, it is still running in 11g R1 Oracle Home." So, something went wrong with the update. For: "BTW, if asmca does not work for me, can I upgrade manually?" - ASMCA is the tool that will perform an ASM upgrade. You can call it interactively, but you will have to use it.
    Support note ID 950200.1 - "ASM 11gR2: How To Upgrade An ASM Instance To Release 11gR2 (11.2.0.1.0)?" describes an ASM upgrade to 11.2. Maybe this note can help you and if it is only to analyze what stage your cluster is in right now. It sounds like, this would be first thing that we need to find out.
    Good luck. Thanks,
    Markus

Maybe you are looking for

  • K7N2 Delta-L: sound software utility?

    K7N2 Delta-L: sound software utility? I want to set the back panel to 6 or atleast 4 channel audio, however the manual says that after you've run the nForce driver setup there should be a software utility you can run to set onboard audio options.  Th

  • I purchased a song, and it won't play. How do I fix it?

    How do I go about getting a song I payed for to actually play?????  I use the same computer or phone to make my purchases, and this has never happened, every other song I downloaded the same night dowloaded correctly, but this one I guess didn't. How

  • Can I replace my 60GB hard drive with this 250GB hard drive?

    Hi all, I've searched the forums and I do not see any mention of the suitability of a 250GB hard drive as a replacement for a 60GB hard drive in a Titanium Powerbook DVI 1GHz. Here is a link: http://www.ifixit.com/Apple-Parts/250-GB-5400-RPM-Western-

  • Safari slow after migration

    Been having problems with load times and rendering after I migrated to same model laptop. Symptoms usually occur when loading multiple tabs and include: - Tabs not loading - Tabs not rendering - Safari hanging - Tabs / pages crashing - Force reload d

  • Almost all of my effects and transitions are not working on my fcpx trial

    i don't know what happened or how to fix it, its getting extremely annoying now. it has happened before then i fixed it and i down really know how i just turned off my computer and it worked then a few days later i turned it off and turned it on it w