RAC Node down and ORA-12514

I have a two node rac setup. One Node went down because of hardware issues. And it seems that I cannot connect from client (jdbc) when SCAN gives particular ip.
I receive : ORA-12514, TNS:listener does not currently know of service requested in connect descriptor. If DNS returns the correct ip - everything works fine.
connection string:
jdbc:oracle:thin:@(DESCRIPTION= (LOAD_BALANCE=on) (ADDRESS=(PROTOCOL=TCP)(HOST=testracscan.internal.int)(PORT=1521)) (CONNECT_DATA=(SERVICE_NAME=testdb.internal.int)))
Interfaces show that VIPS and SCANS are assigned correctly on Node 1:
vlan65 Link encap:Ethernet HWaddr 2C:76:8A:4F:B5:CC
inet addr:192.168.2.10 Bcast:192.168.2.255 Mask:255.255.255.0
inet6 addr: fe80::2e76:8aff:fe4f:b5cc/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:937195 errors:0 dropped:0 overruns:0 frame:0
TX packets:852745 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:186434457 (177.7 MiB) TX bytes:141217705 (134.6 MiB)
vlan65:1 Link encap:Ethernet HWaddr 2C:76:8A:4F:B5:CC
inet addr:192.168.2.25 Bcast:192.168.2.255 Mask:255.255.255.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
vlan65:2 Link encap:Ethernet HWaddr 2C:76:8A:4F:B5:CC
inet addr:192.168.2.35 Bcast:192.168.2.255 Mask:255.255.255.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
vlan65:3 Link encap:Ethernet HWaddr 2C:76:8A:4F:B5:CC
inet addr:192.168.2.30 Bcast:192.168.2.255 Mask:255.255.255.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
vlan65:4 Link encap:Ethernet HWaddr 2C:76:8A:4F:B5:CC
inet addr:192.168.2.110 Bcast:192.168.2.255 Mask:255.255.255.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
vlan65:5 Link encap:Ethernet HWaddr 2C:76:8A:4F:B5:CC
inet addr:192.168.2.115 Bcast:192.168.2.255 Mask:255.255.255.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
[oracle@srvtestdb1 ~]$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.3.0 - Production on 03-SEP-2012 15:35:05
Copyright (c) 1991, 2011, Oracle. All rights reserved.
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.3.0 - Production
Start Date 29-AUG-2012 15:52:57
Uptime 4 days 23 hr. 42 min. 7 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/11.2.0/grid/network/admin/listener.ora
Listener Log File /u01/app/grid/diag/tnslsnr/srvtestdb1/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.2.10)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.2.110)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "testdb.internal.int" has 1 instance(s).
Instance "testdb1", status READY, has 1 handler(s) for this service...
Service "testdbXDB.internal.int" has 1 instance(s).
Instance "testdb1", status READY, has 1 handler(s) for this service...
Service "testdbsvc.internal.int" has 1 instance(s).
Instance "testdb1", status READY, has 1 handler(s) for this service...
The command completed successfully
[oracle@srvtestdb1 ~]$
SQL> show parameter listener
NAME TYPE VALUE
listener_networks string
local_listener string (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.2.110)(PORT=1521))))
remote_listener string testracscan.internal.int:1521
nslookup testracscan.internal.int
Server: 192.168.0.18
Address: 192.168.0.18#53
Name: testracscan.internal.int
Address: 192.168.2.30
Name: testracscan.internal.int
Address: 192.168.2.25
Name: testracscan.internal.int
Address: 192.168.2.35
Problems arise when client ip is resolved to 192.168.2.35 - i get ORA12514.
When IP is resolved to 192.168.2.110 it simply sits ant waits for a moment and then begins to work, and nestat shows:
tcp 0 0 ::ffff:1 192.168.2.5:51685 ::ffff:192.168.2.110:1521 ESTABLISHED
What might be causing this?

[grid@srvtestdb1 ~]$ ps -ef|grep tns
root 65 2 0 Aug29 ? 00:00:00 [netns]
grid 4449 1 0 Aug29 ? 00:00:25 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN2 -inherit
grid 4454 1 0 Aug29 ? 00:00:23 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN3 -inherit
grid 4481 1 0 Aug29 ? 00:00:33 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
grid 37028 1 0 09:38 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
grid 37901 36372 0 09:45 pts/0 00:00:00 grep tns
[grid@srvtestdb1 ~]$
[grid@srvtestdb1 ~]$ srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521
[grid@srvtestdb1 ~]$
[grid@srvtestdb1 ~]$ srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node srvtestdb1
SCAN Listener LISTENER_SCAN2 is enabled
SCAN listener LISTENER_SCAN2 is running on node srvtestdb1
SCAN Listener LISTENER_SCAN3 is enabled
SCAN listener LISTENER_SCAN3 is running on node srvtestdb1
[grid@srvtestdb1 ~]$ srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node srvtestdb1
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node srvtestdb1
SCAN VIP scan3 is enabled
SCAN VIP scan3 is running on node srvtestdb1

Similar Messages

ORA-12505, TNS:listener and ORA-12514: TNS:listener

All,
Connecting to the database through the SQL Developer is giving me the error "ORA-12505, TNS:listener does not currently know of SID given in connect descriptor"
Database version : Oracle 10g XE
O/S : Windows 7, 32 bit
I have restarted my machine and tried but no luck. Tried from sqlplus also and the error was
ERROR:
ORA-12514: TNS:listener does not currently know of service requested in connect
descriptor"
Please provide me the steps to check and resolve this issue.
Thank you very much in advance.
Thanks

Here is what I have in my tnsnames.ora file
XE =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = MSPL-08-09-D169.MSFTPNQ.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = XE)
EXTPROC_CONNECTION_DATA =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC_FOR_XE))
(CONNECT_DATA =
(SID = PLSExtProc)
(PRESENTATION = RO)
ORACLR_CONNECTION_DATA =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC_FOR_XE))
(CONNECT_DATA =
(SID = CLRExtProc)
(PRESENTATION = RO)
Database name is 'XE'

Oracle 10g XE and ORA-12514

Hi,
I just installed Oracle 10g XE on my computer (WinXP Home) and tried to use sqlplus to connect to the db by entering the following standard command:
sqlplus system/system@XEUnfortunately I got ORA-12514 error
ERROR:
ORA-12154: TNS:could not resolve the connect identifier specified
but then sqlplus prompts me once again for user name and pass (it does not prompt for service name) and after entering those two connection is established as shown below
Enter user-name: system
Enter password: <hidden_pass=system>
Connected to:
Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
Calling the command
lsnrctl servicesgives me the following result
LSNRCTL for 32-bit Windows: Version 10.2.0.1.0 - Production on 06-JUN-2007 21:51:00
Copyright (c) 1991, 2005, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC_FOR_XE)))
Services Summary...
Service "CLRExtProc" has 1 instance(s).
Instance "CLRExtProc", status UNKNOWN, has 1 handler(s) for this service...
Handler(s):
"DEDICATED" established:0 refused:0
LOCAL SERVER
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Handler(s):
"DEDICATED" established:0 refused:0
LOCAL SERVER
The command completed successfully
I have no idea why the XE service is unavailable while it is properly configured in my tnsnames.ora file.
I will appreciate your help and advices on how to configure the database to be able to connect to it by providing the service name.
Best regards
Mans

Hm, 12514 or 12154?
ORA-12154: TNS:could not resolve the connect
identifier specifiedThis means that your tnsnames.ora (or other naming method) is missing the called for entry/stanza.
E.g.
C:\>sqlplus u/p@an_alias_not_defined
SQL*Plus: Release 10.2.0.1.0 - Production on Wed Jun 6 23:02:34 2007
Copyright (c) 1982, 2005, Oracle. All rights reserved.
ERROR:
ORA-12154: TNS:could not resolve the connect identifier specified
http://download.oracle.com/docs/cd/B19306_01/network.102/b14212/troublestng.htm#BABBBGGG
http://download.oracle.com/docs/cd/B19306_01/server.102/b14219/tnsus.htm#sthref14854
>
but then sqlplus prompts me once again for user name
and pass (it does not prompt for service name) and
after entering those two connection is established as
shown belowIn the username field you could input for example user@service
If this is the only oracle product installed on the machine, and you are connecting to it locally, you could use just
sqlplus system/system
which uses the "default" instance (on Windows, that is).
lsnrctl services
I have no idea why the XE service is unavailable
while it is properly configured in my tnsnames.oralsnrctl serv is on the server side, it does not matter what you have in tnsnames.ora
Normally the database should register itself to the listener, so the XE instance should have a service for it listed.
If you restart the instance's Windows service, does it show up in the listener?
Message was edited by:
orafad

Oracle 10g ora 12560 and ora 12514 errors

i am a new oracle user. and i have some problems
i installed 10g then,
when i was creating a database,
i received an error messagge "ora-12560",
therefore i cancelled creating database.
i run sqlplus, i received error messagge again
but this messagge differ from first one. it is "ora-12514"
and i tested net manager my oranokta in tnsnames,
same problem is continuesly
and last thing i have a toshiba notebook with intel centrino 1.6
and windows xp with sp2.
i attached below my tnsnames, listener and sqlnet.ora files.
listener.ora
# listener.ora Network Configuration File: D:\oracle\product\10.1.0\db_1\network\admin\listener.ora
# Generated by Oracle configuration tools.
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = D:\oracle\product\10.1.0\db_1)
(PROGRAM = extproc)
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = muttim)(PORT = 1521))
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC))
sqlnet.ora
# sqlnet.ora Network Configuration File: D:\oracle\product\10.1.0\db_1\network\admin\sqlnet.ora
# Generated by Oracle configuration tools.
SQLNET.AUTHENTICATION_SERVICES= (NTS)
NAMES.DIRECTORY_PATH= (TNSNAMES)
tnsnames.ora
# tnsnames.ora Network Configuration File: D:\oracle\product\10.1.0\db_1\network\admin\tnsnames.ora
# Generated by Oracle configuration tools.
ORA10 =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = muttim)(PORT = 1521))
(CONNECT_DATA =
(SERVICE_NAME = Ora10)
ORANOKTA =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = muttim)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = OraNokta)
thank you for help

hi oradba
i opened
start=>run=>services.msc
i didn't find "oracleservice<sid>"
and
i found dbca logfiles,
i attached them
cloneDBCreation.log
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0641: "EXECUTE" sunucu bağlantısı gerektirir
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
ERROR:
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
CloneRmanRestore.log
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
ERROR:
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
lockAccount.log
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
postDBCreation.log
ERROR:
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
SP2-0640: Bağlı değil
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
ERROR:
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0641: "EXECUTE" sunucu bağlantısı gerektirir
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
ERROR:
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
postScripts.log
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
ERROR:
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
ERROR:
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
SP2-0640: Bağlı değil
ERROR:
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
dropping default lexer preference...
SP2-0640: Bağlı değil
dropping default wordlist preference...
SP2-0640: Bağlı değil
dropping default stoplist preference...
SP2-0640: Bağlı değil
dropping default policy...
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
lang_abbr değerini girin: SP2-0310: "D:\oracle\product\10.1.0\db_1\ctx\admin\defaults\drdefset.sql" dosyası açılamıyor
ERROR:
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
SP2-0640: Bağlı değil
SP2-0641: "EXECUTE" sunucu bağlantısı gerektirir
SP2-0640: Bağlı değil
rmanRestoreDatafiles.sql
set echo off;
set serveroutput on;
select systimestamp from dual;
variable devicename varchar2(255);
declare
omfname varchar2(512) := NULL;
done boolean;
begin
dbms_output.put_line(' ');
dbms_output.put_line(' Allocating device.... ');
dbms_output.put_line(' Specifying datafiles... ');
:devicename := dbms_backup_restore.deviceAllocate;
dbms_output.put_line(' Specifing datafiles... ');
dbms_backup_restore.restoreSetDataFile;
dbms_backup_restore.restoreDataFileTo(4, 'D:\oracle\product\10.1.0\oradata\OraNoktaDB\USERS01.DBF', 0, 'USERS');
dbms_backup_restore.restoreDataFileTo(3, 'D:\oracle\product\10.1.0\oradata\OraNoktaDB\SYSAUX01.DBF', 0, 'SYSAUX');
dbms_backup_restore.restoreDataFileTo(2, 'D:\oracle\product\10.1.0\oradata\OraNoktaDB\UNDOTBS01.DBF', 0, 'UNDOTBS1');
dbms_backup_restore.restoreDataFileTo(1, 'D:\oracle\product\10.1.0\oradata\OraNoktaDB\SYSTEM01.DBF', 0, 'SYSTEM');
dbms_output.put_line(' Restoring ... ');
dbms_backup_restore.restoreBackupPiece('D:\oracle\product\10.1.0\db_1\assistants\dbca\templates\Seed_Database.dfb', done);
if done then
dbms_output.put_line(' Restore done.');
else
dbms_output.put_line(' ORA-XXXX: Restore failed ');
end if;
dbms_backup_restore.deviceDeallocate;
end;
select systimestamp from dual;
ultraSearchCfg.log
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
============== Ultra Search Configuration ==============
============== argument list
WKSYSPW=XXXXXXXX
JDBC_CONNSTR=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=muttim)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=OraNokta)))
LAUNCH_ANYWHERE=false
CONN_STRING=
OH=D:\oracle\product\10.1.0\db_1
JDBCDRIVER_PATH=D:\oracle\product\10.1.0\db_1\jdbc\lib\classes12.zip
JDBCNLS_PATH=D:\oracle\product\10.1.0\db_1\jdbc\lib\nls_charset12.zip
JRE_HOME=D:\oracle\product\10.1.0\db_1\jdk\jre\
JAVA_EXE_PATH=D:\oracle\product\10.1.0\db_1\jdk\bin\java.exe
PSEP=;
CTXHX=D:\oracle\product\10.1.0\db_1\bin\ctxhx.exe
==================
ERROR:
ORA-12560: TNS: Protokol bağdaştırıcısı hatası
SP2-0640: Bağlı değil
============== populating the WK$CRAWLER_CONFIG_DEFAULT table
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
============== load crawler classpath
...... setting the crawler classpath
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
============== updating crawler configuration
SP2-0640: Bağlı değil
SP2-0641: "EXECUTE" sunucu bağlantısı gerektirir
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
...... creating lexer preference
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
SP2-0640: Bağlı değil
...... creating wordlist preference
SP2-0640: Bağlı değil
...... creating storage preference
SP2-0640: Bağlı değil
...... creating stoplist
SP2-0640: Bağlı değil
...... creating section group
SP2-0640: Bağlı değil
...... creating zone sections
SP2-0640: Bağlı değil
...... creating field sections
SP2-0640: Bağlı değil
...... creating filter preference
SP2-0640: Bağlı değil
...... creating datastore preference
SP2-0640: Bağlı değil
... checking for invalid objects... see list below
================== list start
SP2-0640: Bağlı değil
================== list end
============== End of Configuration ==============
note : my operating system's language is turkish.

RAC Node hang and unexpected reboot

Hello friends
We are facing the intermittent issue of node hang and unexpected shutdown of node. This is 2 node rac 10.2.03 running on windows 2003. Here's crsd.log
2009-07-16 17:24:03.058: [ OCRMSG][5252]prom_rpc: CLSC recv failure..ret code 7
2009-07-16 17:24:03.058: [ OCRMSG][5252]prom_rpc: possible OCR retry scenario
2009-07-16 17:24:03.058: [ COMMCRS][5616]clscsendx: (0000000002AF5C60) Physical connection (0000000003892080) not active
2009-07-16 17:24:03.058: [ OCRMSG][5616]prom_rpc: CLSC send failure..ret code 11
2009-07-16 17:24:03.058: [ OCRMSG][5616]prom_rpc: possible OCR retry scenario
2009-07-16 17:24:03.105: [ COMMCRS][5252]clscsendx: (0000000002AF5C60) Connection not active
2009-07-16 17:24:03.105: [ OCRMSG][5252]prom_rpc: CLSC send failure..ret code 6
2009-07-16 17:24:03.105: [ OCRMSG][5252]prom_rpc: possible OCR retry scenario
2009-07-16 17:24:03.105: [ COMMCRS][5616]clscsendx: (0000000002AF5C60) Connection not active
2009-07-16 17:24:03.105: [ OCRMSG][5616]prom_rpc: CLSC send failure..ret code 6
2009-07-16 17:24:03.105: [ OCRMSG][5616]prom_rpc: possible OCR retry scenario
2009-07-16 17:24:03.152: [ COMMCRS][5252]clscsendx: (0000000002AF5C60) Connection not active
2009-07-16 17:24:03.152: [ OCRMSG][5252]prom_rpc: CLSC send failure..ret code 6
2009-07-16 17:24:03.152: [ OCRMSG][5252]prom_rpc: possible OCR retry scenario
2009-07-16 17:24:03.168: [ COMMCRS][5616]clscsendx: (0000000002AF5C60) Connection not active
2009-07-16 17:24:03.168: [ OCRMSG][5616]prom_rpc: CLSC send failure..ret code 6
2009-07-16 17:24:03.168: [ OCRMSG][5616]prom_rpc: possible OCR retry scenario
2009-07-16 17:24:03.215: [ COMMCRS][5252]clscsendx: (0000000002AF5C60) Connection not active
2009-07-16 17:24:03.215: [ OCRMSG][5252]prom_rpc: CLSC send failure..ret code 6
2009-07-16 17:24:03.215: [ OCRMSG][5252]prom_rpc: possible OCR retry scenario
2009-07-16 17:24:03.215: [ COMMCRS][5616]clscsendx: (0000000002AF5C60) Connection not active
2009-07-16 17:24:03.215: [ OCRMSG][5616]prom_rpc: CLSC send failure..ret code 6
2009-07-16 17:24:03.215: [ OCRMSG][5616]prom_rpc: possible OCR retry scenario
2009-07-16 17:24:03.261: [ COMMCRS][5616]clscsendx: (0000000002AF5C60) Connection not active
2009-07-16 17:24:03.261: [ OCRMSG][5616]prom_rpc: CLSC send failure..ret code 6
2009-07-16 17:24:03.261: [ OCRMSG][5616]prom_rpc: possible OCR retry scenario
Please throw me the light, what may be issue.

I suggest you install [ IPD/OS|http://www.oracle.com/technology/products/database/clustering/ipd_download_homepage.html] on you cluster. This will give you all the relevant OS statistics so when a node reboot happens, you can figure out what the state of the nodes was at that time and then fix the problem. The hang is often caused by something other than Oracle RAC.

SQLDeveloper, db links, and ora-12514

There are several ducks to line up here, so please bear with me.
First, the lineup, the batting order …
Workstation WKS_DBA; Win 7 Pro 64bit; Oracle 10g 32bit client; Oracle 11g 64bit client (in that order in PATH); SQL Developer version 3.2.20.09, build MAIN-09.87
Workstation WKS_OLD; Win 7 Pro 32bit; Oracle 10g 32bit client; SQL Developer version 3.0.04, build MAIN-04.34
Workstation WKS-NEW; Win 7 Pro 64bit; 11g 32bit client; 11g 64bit client (in that order in PATH).
Database server SVRLCL; Oracle Linux 5.6 x86-64; Oracle database 10.2.0.4 64bit (let’s call it DBLCL); database link LK_DBRMT pointing to ..
Database server SVRRMT; Windows server 2003 R2, SP2, 32bit; Oracle database 10.2.0.4 (let’s call it DBRMT).
The situation: Connected to DBLCL, we have an UPDATE statement with a WHERE clause that includes a SELECT on a table @LK_DBRMT.
If the statement is executed by SQLDeveloper on WKS_DBA, it works.
If the statement is executed by SQLDeveloper on WKS_OLD, it works.
If the statement is executed by SQLDeveloper on WKS_NEW, it returns an ora-12514, pointing to the db link.
What’s equally strange is that from WKS_OLD and WKS_NEW, SQLDev is sending some CREATE DATABASE LINK statements prior to sending the UPDATE statement. When coming from WKS_NEW, that supposed db link creation includes SERVICE_NAME=null. Here is a sequence of packets received by database DBLCL from SQLDev, beginning with the last query that SQLDev issued on its own behalf, the select from dba.xml_schemas. The only statement we asked sqldev to execute was the UPDATE.
(obviously, the trace has been edited in several ways for brevity and readability. Actualy packet contents include only key clear text to identify what was going on.)
15:02:26:948]     nttrd:      socket 24 had bytes read=112
15:02:26:948]     nsprecv:    |..select.1.from.dba_xml_schemas.|
15:02:26:949]     nttwr:      socket 24 had bytes written=159
15:02:26:949]     nspsend:    |........|
15:02:31:124]     nttrd:      socket 24 had bytes read=277
15:02:31:124]     nsprecv:      |....CREATE.DATABASE.LINK.LK_DBRMT..CONNECT.TO.kilroy.IDENTIFIED.BY.mypswd..USING.'(DESCRIPTION.=(ADDRESS_LIST.=(ADDRESS.=.(PROTOCOL.=.TCP)(HOST.=.172.nn.nnn.88)(PORT.=.1521)))(CONNECT_DATA.=.(SERVICE_NAME.=.null))).'..|
15:02:31:128]     nttwr:      socket 24 had bytes written=60
15:02:31:128]     nspsend:    |.<............................D.............................    |
15:02:31:129]     nttrd:      socket 24 had bytes read=269
15:02:31:129]     nsprecv:      |....CREATE.DATABASE.LINK.LK_DBRMT..CONNECT.TO.kilroy.IDENTIFIED.BY.mypswd..USING.'(DESCRIPTION.=(ADDRESS_LIST.=(ADDRESS.=.(PROTOCOL.=.TCP)(HOST.=.172.nn.nnn.88)(PORT.=.1521)))(CONNECT_DATA.=.(SERVICE_NAME.=.null))).'..|
15:02:31:129]     nttwr:      socket 24 had bytes written=11
15:02:31:129]     nspsend:    |...........     |
15:02:31:130]     nttwr:      socket 24 had bytes written=11
15:02:31:130]     nspsend:    |...........     |
15:02:31:130]     nttrd:      socket 24 had bytes read=11
15:02:31:130]     nsprecv:    |...........     |
15:02:31:130]     nttwr:      socket 24 had bytes written=84
15:02:31:130]     nspsend:    |...(ORA-02011:.duplicate.database.link.name.    |
15:02:31:131]     nttrd:      socket 24 had bytes read=269
15:02:31:131]     nsprecv:      |....CREATE.DATABASE.LINK.LK_DBRMT..CONNECT.TO.kilroy.IDENTIFIED.BY.mypswd..USING.'(DESCRIPTION.=(ADDRESS_LIST.=(ADDRESS.=.(PROTOCOL.=.TCP)(HOST.=.172.nn.nnn.88)(PORT.=.1521)))(CONNECT_DATA.=.(SERVICE_NAME.=.null))).'..|
15:02:31:132]     nttwr:      socket 24 had bytes written=11
15:02:31:132]     nspsend:    |...........     |
15:02:31:132]     nttwr:      socket 24 had bytes written=11
15:02:31:132]     nspsend:    |...........     |
15:02:31:132]     nttrd:      socket 24 had bytes read=11
15:02:31:132]     nsprecv:    |...........     |
15:02:31:132]     nttwr:      socket 24 had bytes written=84
15:02:31:132]     nspsend:    |...(ORA-02011:.duplicate.database.link.name.    |
15:02:31:134]     nttrd:      socket 24 had bytes read=848
15:02:31:134]     nsprecv:    |.....update.TRANS_DETAIL.RT.|
15:02:31:138]     nttwr:      socket 11 had bytes written=227
15:02:31:138]     nspsend:      |..(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=172.nn.nnn.88)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=null)(CID=(PROGRAM=oracle)(HOST=svrlcl.myorg.org)(USER=KIlroy))))     |
15:02:31:138]     nttrd:      socket 11 had bytes read=103
15:02:31:138]     nsprecv:      "|""..[(DESCRIPTION=(TMP=)(VSNNUM=169870336)(ERR=12514)(ERROR_STACK=(ERROR=(CODE=12514)(EMFI=4)))) |
15:02:31:139]     nttwr:      socket 24 had bytes written=11
15:02:31:139]     nspsend:    |...........     |
15:02:31:139]     nttwr:      socket 24 had bytes written=11
15:02:31:140]     nspsend:    |...........     |
15:02:31:140]     nttrd:      socket 24 had bytes read=11
15:02:31:140]     nsprecv:    |...........     |
15:02:31:140]     nttwr:      socket 24 had bytes written=136
15:02:31:140]     nspsend:    |....[ORA-12514:.TNS:listener.does.not.currently.know.of.service.requested.in.connect.descriptor.|
Here is the same sequence, using WKS_OLD
09:25:34:694]     nttrd:      socket 24 had bytes read=112
09:25:34:694]     nsprecv:    |..select.1.from.dba_xml_schemas.|
09:25:34:694]     nttwr:      socket 24 had bytes written=159
09:25:34:694]     nspsend:    |........|
09:25:39:883]     nttrd:      socket 24 had bytes read=279
09:25:39:883]     nsprecv:      |....CREATE.DATABASE.LINK.LK_DBRMT..CONNECT.TO.kilroy.IDENTIFIED.BY.mypswd..USING.'(DESCRIPTION.=(ADDRESS_LIST.=(ADDRESS.=.(PROTOCOL.=.TCP)(HOST.=.172.nn.nnn.88)(PORT.=.1521)))(CONNECT_DATA.=.(SERVICE_NAME.=.DBRMT))).'|
09:25:39:979]     nttwr:      socket 24 had bytes written=60
09:25:39:979]     nspsend:    |.<......|
09:25:39:980]     nttrd:      socket 24 had bytes read=271
09:25:39:980]     nsprecv:      |....CREATE.DATABASE.LINK.LK_DBRMT..CONNECT.TO.kilroy.IDENTIFIED.BY.mypswd..USING.'(DESCRIPTION.=(ADDRESS_LIST.=(ADDRESS.=.(PROTOCOL.=.TCP)(HOST.=.172.nn.nnn.88)(PORT.=.1521)))(CONNECT_DATA.=.(SERVICE_NAME.=.DBRMT))).'|
09:25:39:981]     nttwr:      socket 24 had bytes written=11
09:25:39:981]     nspsend:    |...........     |
09:25:39:981]     nttwr:      socket 24 had bytes written=11
09:25:39:981]     nspsend:    |...........     |
09:25:39:981]     nttrd:      socket 24 had bytes read=11
09:25:39:981]     nsprecv:    |...........     |
09:25:39:982]     nttwr:      socket 24 had bytes written=84
09:25:39:982]     nspsend:    |...(ORA-02011:.duplicate.database.link.name.    |
09:25:39:983]     nttrd:      socket 24 had bytes read=271
09:25:39:983]     nsprecv:      |....CREATE.DATABASE.LINK.LK_DBRMT..CONNECT.TO.kilroy.IDENTIFIED.BY.mypswd..USING.'(DESCRIPTION.=(ADDRESS_LIST.=(ADDRESS.=.(PROTOCOL.=.TCP)(HOST.=.172.nn.nnn.88)(PORT.=.1521)))(CONNECT_DATA.=.(SERVICE_NAME.=.DBRMT))).'|
09:25:39:984]     nttwr:      socket 24 had bytes written=11
09:25:39:984]     nspsend:    |........... |
09:25:39:985]     nttwr:      socket 24 had bytes written=11
09:25:39:985]     nspsend:    |...........     |
09:25:39:985]     nttrd:      socket 24 had bytes read=11
09:25:39:985]     nsprecv:    |...........     |
09:25:39:985]     nttwr:      socket 24 had bytes written=84
09:25:39:986]     nspsend:    |...(ORA-02011:.duplicate.database.link.name.    |
09:25:39:988]     nttrd:      socket 24 had bytes read=864
09:25:39:988]     nsprecv:    |.....update.ROY_TRANS_DETAIL.RT.|
09:25:40:015]     nttwr:      socket 12 had bytes written=229
09:25:40:015]     nspsend:      |..(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=172.nn.nnn.88)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=DBRMT)(CID=(PROGRAM=oracle)(HOST=svrlcl.myorg.org)(USER=KIlroy))))   |
09:25:40:020]     nttrd:      socket 12 had bytes read=8
09:25:40:020]     nsprecv:    |........|
09:25:40:020]     nttwr:      socket 12 had bytes written=229
09:25:40:020]     nspsend:      |..(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=172.nn.nnn.88)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=DBRMT)(CID=(PROGRAM=oracle)(HOST=svrlcl.myorg.org)(USER=KIlroy))))   |
09:25:40:021]     nttrd:      socket 12 had bytes read=32
09:25:40:021]     nsprecv:    |........|
09:25:40:021]     nttwr:      socket 12 had bytes written=38
09:25:40:021]     nspsend:    |..x86_64/Linux.2.4.xx. |
09:25:40:021]     nttrd:      socket 12 had bytes read=179
09:25:40:021]     nsprecv:    |.....IBMPC/WIN_NT-8.1.0.|
And the same sequence when executing from WKS_DBA
[10-JUL-2013      14:34:24:546]     nttrd:      socket 24 had bytes read=111
[10-JUL-2013      14:34:24:546]     nsprecv:    |.select.1.from.dba_xml_schemas.w|
[10-JUL-2013      14:34:24:547]     nttwr:      socket 24 had bytes written=159
[10-JUL-2013      14:34:24:547]     nspsend:    |........|
[10-JUL-2013      14:34:45:262]     nttrd:      socket 24 had bytes read=856
[10-JUL-2013      14:34:45:262]     nsprecv:    |.....update.ROY_TRANS_DETAIL.RT.|
[10-JUL-2013      14:34:45:267]     nttwr:      socket 11 had bytes written=226
[10-JUL-2013      14:34:45:267]     nspsend:      |..(DESCRIPTION=(address=(protocol=tcp)(host=svrrmt)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=DBRMT)(CID=(PROGRAM=oracle)(HOST=svrlcl.myorg.org)(USER=estevens))))      |
[10-JUL-2013      14:34:45:272]     nttrd:      socket 11 had bytes read=8
[10-JUL-2013      14:34:45:272]     nsprecv:
[10-JUL-2013      14:34:45:272]     nsprecv:    |........|
[10-JUL-2013      14:34:45:272]     nttwr:      socket 11 had bytes written=226
[10-JUL-2013      14:34:45:272]     nspsend:      |..(DESCRIPTION=(address=(protocol=tcp)(host=svrrmt)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=DBRMT)(CID=(PROGRAM=oracle)(HOST=svrlcl.myorg.org)(USER=estevens))))      |
[10-JUL-2013      14:34:45:273]     nttrd:      socket 11 had bytes read=32
[10-JUL-2013      14:34:45:273]     nsprecv:    |........|
[10-JUL-2013      14:34:45:273]     nttwr:      socket 11 had bytes written=38
[10-JUL-2013      14:34:45:273]     nspsend:    |..x86_64/Linux.2.4.xx. |
[10-JUL-2013      14:34:45:273]     nttrd:      socket 11 had bytes read=179
[10-JUL-2013      14:34:45:273]     nsprecv:    |.....IBMPC/WIN_NT-8.1.0.|
Several questions suggest themselves, not the least of which are
1- Why is SQLDev even issuing those CREATE DATABASE LINK statements in the first place?
2- Where is it getting the information with which to do that?
3- Why is the same version of SQLDev behaving differently on WKS_DBA and WKS_NEW ?

rp0428 wrote:
>
On WKS_NEW, the UPDATE returns an ora-12514 on the reference to the db_link. Of course the db_link is at the database so at first blush the client should have not bearing.
>
I don't know why sql developer should be sending CREATE DATABASE LINK ddl at all so the dev team should look into this. Gary's attempt to get a response from Vadim didnt' work.
The only 'common' thing I see is that on WKS_NEW you have the 32 bit Oracle client ahead of the 64 bit client; the others don't have that.
Ok, on Friday, I had un-restricted access to the problem workstation (developer was out of office). I completely removed all oracle clients, then started re-installing and testing. I removed and reinstalled only the Oracle clients, not SQL Dev. All removals were 'ripping it out by the roots' - deleting the installation directories (c:\app\oracle etc), the inventory directories (c:\Program Files\oracle and c:\Program Files (x86)\oracle) and the registry entires (hklm\software\oracle and hklm\software\Wow6432Node\oracle).
First I installed just the 32-bit. SQL Dev choked and said 'cannot load 32-bit libraries'
Removed the 32 and installed the 64 bit. Ran the tests and got the same results as before .. SELECT works, SELECT in the UPDATE fails
Removed 64, installed 32 then 64. Ran the tests and got the same results as before .. SELECT works, SELECT in the UPDATE fails
Reversed the PATH entries, putting 32-bit first. Ran the tests and got the same results as before .. SELECT works, SELECT in the UPDATE fails
Which begs the question: what type of connection are you using: OCI or thin? If OCI try using the THIN driver to see if you get the same issue and log entries.
If that selection is made by setting the connection properties as "basic" (thin?) or "tns" (OCI?) then yes, tried it both ways, no change.
As to other things to try I would suggest more tests to try to isolate the offending part a little more.
Test various aspects of the DB_LINK itself to see which use cases work and which do not. Many of these can just be done from WKS_NEW
1. Try a plain SELECT * from myTable@dblink - tests if the DB link itself even works in the FROM clause with NO local table reference.
2. Try a similar SELECT * and add a second remote table in the WHERE clause - that is ALL tables and conditions are from the remote server.
3. Try a SELECT from a local table that joins the remote table and db link in the FROM clause rather than the WHERE clause
I think the original pretty well covers the bases. Here's the actual full update statement (with certain names made generic for public posting)
update LOCAL_TRANS_DETAIL RT
   set rt.sent_to_remoteapp_date = sysdate
where rt.transactionid in
(select rt.transactionid
   FROM LOCAL_TRANS_DETAIL RT
   JOIN LOCALSCHEMA.RMT_MASTER_ITEMS RMI ON RT.NEW_ITEM_NUM = RMI.NEW_ITEM_NUM
   JOIN RMTSCHEMA_A.PAC_PRODUCT_NUMBER@RMTDB PPN ON RT.NEW_ITEM_NUM = PPN.PRODUCT_NUMBER
   JOIN RMTSCHEMA_A.PAC_PRODUCT_NUMBER@RMTDB PPN2 ON PPN.PRODUCT_ID = PPN2.PRODUCT_ID AND PPN2.PRIMARY_SW = 'Y'
   JOIN RMTSCHEMA_B.PRD_IMPRESSION@RMTDB PI ON PI.ORDER_NO = PPN2.PRODUCT_NUMBER
WHERE RT.SENT_TO_remoteapp_DATE is NULL and
      rt.order_id = 'W4011693' and
      rt.item_ship_date < '01-AUG-2013')
4. Try removing the 32 bit Oracle client form the WKS_NEW machine to test if it was getting in the way.
Done. No difference, see previous comment.

10g RAC Node down Enterprise Manager

Hello
We have a 2 node Oracle 10g RAC Rel2 Linux setup. Enterprise Manager was first stalled on Node1 and we access it using http://node1:5500/em.
This node has a hardware failure and is out of commision at the moment. When I try to connect to http://node2.5500/em it does not work.
I see the dbconsole process is running onthe node2.
How can I use the Enterprise Manager if the node1 is down?
Thanks

Can you try deconfig and config again on node 2?
emctl stop dbconsol
emca -deconfig dbcontrol
emca -config dbcontrolSalman

RAC node down

Hi All,
Oracle 10.2.0
Windows server 2003
My doubt is, In the Oracle RAC server if the there are some connections to the node1 and node2 through some applications.
If node1 is down, Will the existing connections to the node1 will be redirected to the node2 ?
Please Advice..
TIA,

If you did configure transparent application failover (TAF) in the tnsnames.ora of the clients, yes.
You can read about TAF in the Net Administrators Manaul at http://tahiti.oracle.com
Sybrand Bakker
Senior Oracle DBA

Rac node evicted and asm related

Hi friends
I have few doubts in rac environment
1.In 2 node rac while adding datafile to tablespace if you forget to metion '+'then what will happen whether it is going to be create or it throws an error if it creates where exactly located and other node users how to work on that tablespace .what all steps to perform that datafile is usefull for all node users.
2. In Rac environment how to check how many sessions connected to particular node.
3)
In Rac any node is evicted due to network failure then after we rebuild the network .Is there any steps to do manually to access the failure node after rebuilding the network or it will automatically available in cluster group which service is perform this activity.
4.While configuring clusterware you choose voting disk and ocr disk location and which redundancy you will choose suppose if you go for normal redundancy how many disks you can select for each file either one or two?.

[grid@srvtestdb1 ~]$ ps -ef|grep tns
root 65 2 0 Aug29 ? 00:00:00 [netns]
grid 4449 1 0 Aug29 ? 00:00:25 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN2 -inherit
grid 4454 1 0 Aug29 ? 00:00:23 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN3 -inherit
grid 4481 1 0 Aug29 ? 00:00:33 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
grid 37028 1 0 09:38 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
grid 37901 36372 0 09:45 pts/0 00:00:00 grep tns
[grid@srvtestdb1 ~]$
[grid@srvtestdb1 ~]$ srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521
[grid@srvtestdb1 ~]$
[grid@srvtestdb1 ~]$ srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node srvtestdb1
SCAN Listener LISTENER_SCAN2 is enabled
SCAN listener LISTENER_SCAN2 is running on node srvtestdb1
SCAN Listener LISTENER_SCAN3 is enabled
SCAN listener LISTENER_SCAN3 is running on node srvtestdb1
[grid@srvtestdb1 ~]$ srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node srvtestdb1
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node srvtestdb1
SCAN VIP scan3 is enabled
SCAN VIP scan3 is running on node srvtestdb1

Private Interconnect: Should any nodes other than RAC nodes have one?

The contractors that set up our four-node production 10g RAC (and a standalone development server) also assigned private interconnect addresses to 2 Apache/ApEx servers and a standalone development database server.
There are service names in the tnsnames.ora on all servers in our infrastructure referencing these private interconnects- even the non-rac member servers. The nics on these servers are not bound for failover with the nics bound to the public/VIP addresses. These nics are isolated on their own switch.
Could this configuration be related to lost heartbeats or voting disk errors? We experience rac node expulsions and even arbitrary bounces (reboots!) of all the rac nodes.

I do not have access to the contractors. . . .can only look at what they have left behind and try to figure out their intention. . .
I am reading the Ault/Tumha book Oracle 10g Grid and Real Application Clusters and looking through our own settings and config files and learning srvctl and crsctl commands from their examples. Also googling and OTN searching through the library full of documentation. . .
I still have yet to figure out if the private interconnect spoken about so frequently in cluster configuration documents are the binding to the set of node.vip address specifications in the tnsnames.ora (bound the the first eth adaptor along with the public ip addresses for the nodes) or the binding on the second eth adaptor to the node.prv addresses not found in the local pfile, in the tnsnames.ora, or the listener.ora (but found at the operating system level in the ifconfig). If the node.prv addresses are not the private interconnect then can anyone tell me that they are for?

ORA-12514 on R12 EBS Apps server when 1 DB RAC node crashed/down

Just now Production 11.2.0.2 RAC DB on windows 2008 server Node1 crashed. While on Node 2 all services are up and running including database. But from EBS R12.1.2 application server when connecting as username/password from sql*plus is throwing ORA-12514 error.
While It is connecting if I give username/password@TNSNAME but not without @TNSNAME. Due to this none of the application services are starting.
Please help/advise. Thank you.
Following is the tnsnames.ora,
# This file is automatically generated by AutoConfig. It will be read and
# overwritten. If you were instructed to edit this file, or if you are not
# able to use the settings created by AutoConfig, refer to Metalink Note
# 387859.1 for assistance.
#$Header: NetServiceHandler.java 120.19.12010000.6 2010/03/09 08:11:36 jmajumde ship $
ORCL=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orcldbscan)(PORT=1521))
            (CONNECT_DATA=
                (SERVICE_NAME=ORCL)
                (INSTANCE_NAME=ORCL1)
ORCL1=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orcldbscan)(PORT=1521))
            (CONNECT_DATA=
                (SERVICE_NAME=ORCL)
                (INSTANCE_NAME=ORCL1)
ORCL1_FO=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orcldbscan)(PORT=1521))
            (CONNECT_DATA=
                (SERVICE_NAME=ORCL)
                (INSTANCE_NAME=ORCL1)
ORCL_FO=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orcldbscan)(PORT=1521))
            (CONNECT_DATA=
                (SERVICE_NAME=ORCL)
                (INSTANCE_NAME=ORCL1)
ORCL1=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orcldbscan)(PORT=1521))
            (CONNECT_DATA=
                (SERVICE_NAME=ORCL)
                (INSTANCE_NAME=ORCL1)
ORCL1_FO=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orcldbscan)(PORT=1521))
            (CONNECT_DATA=
                (SERVICE_NAME=ORCL)
                (INSTANCE_NAME=ORCL1)
ORCL2=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orcldb2-vip.sa.company.net)(PORT=1521))
            (CONNECT_DATA=
                (SERVICE_NAME=ORCL)
                (INSTANCE_NAME=ORCL2)
ORCL2_FO=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orcldb2-vip.sa.company.net)(PORT=1521))
            (CONNECT_DATA=
                (SERVICE_NAME=ORCL)
                (INSTANCE_NAME=ORCL2)
ORCL_BALANCE=
        (DESCRIPTION=
            (ADDRESS_LIST=
                (LOAD_BALANCE=YES)
                (FAILOVER=YES)
                (ADDRESS=(PROTOCOL=tcp)(HOST=orcldb1-vip.sa.company.net)(PORT=1521))
                (ADDRESS=(PROTOCOL=tcp)(HOST=orcldb2-vip.sa.company.net)(PORT=1521))
            (CONNECT_DATA=
                (SERVICE_NAME=ORCL)
FNDFS_orclAPPL=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orclAPPL.sa.company.net)(PORT=1626))
            (CONNECT_DATA=
                (SID=FNDFS)
FNDFS_orclAPPL.sa.company.net=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orclAPPL.sa.company.net)(PORT=1626))
            (CONNECT_DATA=
                (SID=FNDFS)
FNDFS_ORCL_orclAPPL=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orclAPPL.sa.company.net)(PORT=1626))
            (CONNECT_DATA=
                (SID=FNDFS)
FNDFS_ORCL_orclAPPL.sa.company.net=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orclAPPL.sa.company.net)(PORT=1626))
            (CONNECT_DATA=
                (SID=FNDFS)
FNDSM_orclAPPL_ORCL=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orclAPPL.sa.company.net)(PORT=1626))
            (CONNECT_DATA=
                (SID=FNDSM)
FNDSM_orclAPPL.sa.company.net_ORCL=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orclAPPL.sa.company.net)(PORT=1626))
            (CONNECT_DATA=
                (SID=FNDSM)
FNDFS_APPLTOP_orclappl=
        (DESCRIPTION=
            (ADDRESS_LIST=
                (ADDRESS=(PROTOCOL=tcp)(HOST=orclAPPL.sa.company.net)(PORT=1626))
            (CONNECT_DATA=
                (SID=FNDFS)
IFILE=E:\ORACLE\ORCL\INST\APPS\ORCL_orclappl\ora\10.1.2\network\admin\ORCL_orclappl_ifile.ora

A database/OS crash shouldn't have such impact on your configuration. Please review the following docs and verify your setup.
Using Oracle 11g Release 2 Real Application Clusters with Oracle E-Business Suite Release 12 (Doc ID 823587.1)
Configuring and Managing E-Business Application Tier for RAC (Doc ID 1311528.1)
Thanks,
Hussein

Node 2 in RAC 11gr2 went down and hasn't been able to restart. Help pls.

Hello all,
I'm working on this RAC configuration 11gR2 updated to the last patchset. During the night everything on node 2 went down and hasn't been able to come back, not even after a reboot. I've checked through the logfiles and I think the problem might be traceable to this:
vi ocssd.log2013-01-15 02:42:47.629: [    CSSD][1104660800]clssnmvDHBValidateNcopy: node 1, dbacc1, has a disk HB, but no network HB, DHB has rcfg 247316625, wrtcnt, 18953467, LATS 165822984, lastSeqNo 18949942, uniqueness 1357546658, timestamp 1358214167/666569324
… after some time...
2013-01-15 02:43:01.929: [    CSSD][1115699520]###################################
2013-01-15 02:43:01.929: [    CSSD][1115699520]clssscExit: CSSD aborting from thread clssnmRcfgMgrThread
2013-01-15 02:43:01.929: [    CSSD][1115699520]###################################
2013-01-15 02:43:01.929: [    CSSD][1115699520](:CSSSC00012:)clssscExit: A fatal error occurred and the CSS daemon is terminating abnormally.
crsctl check cluster -all**************************************************************
dbacc1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
dbacc2:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
I also tried to ping the vip from one machine to the other and vice versa and they all answered fine, also with huge size ping (i've found that this could have spotted a possibile bug).
Any help would be really really appreciated. Thanks.
Edited by: Klawd on 16-gen-2013 9.29

crsctl stat res -t -init on node 1
NAME           TARGET STATE        SERVER                 STATE_DETAILS
Cluster Resources
ora.asm
      1        ONLINE ONLINE       dbacc1                 Started
ora.cluster_interconnect.haip
      1        ONLINE ONLINE       dbacc1
ora.crf
      1        ONLINE ONLINE       dbacc1
ora.crsd
      1        ONLINE ONLINE       dbacc1
ora.cssd
      1        ONLINE ONLINE       dbacc1
ora.cssdmonitor
      1        ONLINE ONLINE       dbacc1
ora.ctssd
      1        ONLINE ONLINE       dbacc1                 OBSERVER
ora.diskmon
      1        OFFLINE OFFLINE
ora.drivers.acfs
      1        ONLINE ONLINE       dbacc1
ora.evmd
      1        ONLINE ONLINE       dbacc1
ora.gipcd
      1        ONLINE ONLINE       dbacc1
ora.gpnpd
      1        ONLINE ONLINE       dbacc1
ora.mdnsd
      1        ONLINE ONLINE       dbacc1crsctl stat res -t -init on node 2
NAME           TARGET STATE        SERVER                 STATE_DETAILS
Cluster Resources
ora.asm
      1        ONLINE OFFLINE
ora.cluster_interconnect.haip
      1        ONLINE OFFLINE
ora.crf
      1        ONLINE ONLINE       dbacc2
ora.crsd
      1        ONLINE OFFLINE
ora.cssd
      1        ONLINE OFFLINE
ora.cssdmonitor
      1        ONLINE ONLINE       dbacc2
ora.ctssd
      1        ONLINE OFFLINE
ora.diskmon
      1        OFFLINE OFFLINE
ora.drivers.acfs
      1        ONLINE ONLINE       dbacc2
ora.evmd
      1        ONLINE OFFLINE
ora.gipcd
      1        ONLINE ONLINE       dbacc2
ora.gpnpd
      1        ONLINE ONLINE       dbacc2
ora.mdnsd
      1        ONLINE ONLINE       dbacc2Edited by: Klawd on 16-gen-2013 11.49
Edited by: Klawd on 16-gen-2013 12.18

Changed IP's of nodes cause ORA-12514

Hi ,
we reactivated an older RAC (2 nodes) and ran into the following situation:
- the original IP's of the scan-address have been reused, therefore we had to use new IP's
After confuguring the new IP's the command <nslookup> myrac-scan.de.domain.dns returns this result:
Server:         10.xx.xx.x
Address:      10.xx.xx.x#53
Name:   myrac-scan.de.domain.dns
Address: 10.xxx.x.213
Name:   myrac-scan.de.domain.dns
Address: 10.xxx.x.212
Name:   myrac-scan.de.domain.dns
Address: 10.xxx.x.211
Executing <srvctl config scan> returns this:
SCAN name: myrac-scan.de.domain.dns, Network: 1/10.137.1.0/255.255.255.0/bond0
SCAN VIP name: scan1, IP: /myrac-scan.de.domain.dns/10.xxx.x.212
SCAN VIP name: scan2, IP: /myrac-scan.de.domain.dns/10.xxx.x.211
SCAN VIP name: scan3, IP: /myrac-scan.de.domain.dns/10.xxx.x.213
Therefore: everything looks good to us!
When executing <lsnrctl services> we get this:
Service "RAC_DB.de.domain.dns" has 1 instance(s).
Instance "RAC02", status READY, has 1 handler(s) for this service...
    Handler(s):
      "DEDICATED" established:19 refused:0 state:ready
         LOCAL SERVER
Testing the service RAC_DB.de.domain.dns works fine, too.
But when trying to connect from my PC (or any other client) we are getting the error ORA-12514 TNS:listener does not currently know of service requested in connect descriptor
My TNSNAMES.ORA looks like this:
RAC.de.domain.dns =
(DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = myrac-scan.de.domain.dns)(PORT = 1521))
    (CONNECT_DATA =
      (SERVICE_NAME = RAC_DB.de.domain.dns)
      (SERVER = DEDICATED)
Any idea? When using PUTTY with myrac-scan.de.domain.dns I get connected to one of the nodes of the RAC - therefore the address itself is OK.
If the IP's have changed: is ther anything else which needs to be reconfigured?
Any help will be appreciated.
Rgds
Jan

Does the SCAN listener show any registered services from the RAC db instances?
If not, then it means that the registration process fails - in which case you need to check things like hostname resolution and firewalls on the RAC server nodes.
If the SCAN listener does show RAC services, confirm that the service name used by the client's TNS connection is supported.
If there still does not seem to be an error, enable SQL*Net tracing on the client. Set the client's sqlnet.ora file to include the following settings:
DIAG_ADR_ENABLED = off
TRACE_LEVEL_CLIENT = admin
TRACE_DIRECTORY_CLIENT = <your-local-trace-dir>
Try connecting from the client again and investigate the trace for that connection attempt.

Ora-12514 connect to SCAN listener in dataguard RAC environment

I have two rac nodes datase on linux and two rac standby database also.
Here are what I got when I try to connect to scan listener from client. It connects to vip/or scan ip addresses without any problem.
sqlplus system/password@wxcy
SQL*Plus: Release 10.2.0.3.0 - Production on Wed Mar 14 18:42:42 2012
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
ERROR:
ORA-12154: TNS:could not resolve the connect identifier specified
But when I log in to the ip address of the scan listener, it worked:
sqlplus system/[email protected]:1521/wxcy
SQL*Plus: Release 10.2.0.3.0 - Production on Wed Mar 14 18:43:12 2012
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL>
Here is the tnsnames.ora on the client:
wxcy =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)
(HOST = scan.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = wxcy)
Here are listener.ora and tnsnames.ora on the server:
LISTENER=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))) # line added by Agent
LISTENER_SCAN3=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN3)))) # line added by Agent
LISTENER_SCAN2=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN2)))) # line added by Agent
LISTENER_SCAN1=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))) # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN1=ON # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN2=ON # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN3=ON # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER=ON # line added by Agent
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = wxcy_DGMGRL)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
(SID_NAME = wxcy1)
SID_LIST_LISTENER_SCAN1 =
(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = wxcy)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
(SID_NAME = wxcy1)
SID_LIST_LISTENER_SCAN3 =
(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = wxcy)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
(SID_NAME = wxcy1)
INBOUND_CONNECT_TIMEOUT_LISTENER = 120
~
tnsnames.ora:
wxcy =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = scan.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = wxcy)
wxcy_DGMGRL =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = scan.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = wxcy_DGMGRL)
wxcy1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = scan.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)(SERVICE_NAME = wxcy)
(INSTANCE_NAME = wxcy1)
wxcy2 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = scan.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)(SERVICE_NAME = wxcy)
(INSTANCE_NAME = wxcy2)
From node 2:
LISTENER_SCAN2=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN2)))) # line added by Agent
LISTENER_SCAN3=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN3)))) # line added by Agent
LISTENER=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))) # line added by Agent
LISTENER_SCAN1=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))) # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN1=ON # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER=ON # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN3=ON # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN2=ON # line added by Agent
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = wxcy_DGMGRL)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
(SID_NAME = wxcy1)
SID_LIST_LISTENER_SCAN2 =
(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = wxcy)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
(SID_NAME = wxcy2)
SID_LIST_LISTENER_SCAN3 =
(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = wxcy)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
(SID_NAME = wxcy2)
I wonder why I can connect to the ip addresses of those scan listener but not the scan name.
Thanks.

Hello;
This note may help :
How to Troubleshoot Connectivity Issue with 11gR2 SCAN Name [ID 975457.1]
Also I would check my DNS ( OK double check ) and each listener service as follows :
lsnrctl service LISTENER_SCAN1
( and so on )
Also
Make sure a password file exists on on that node. ( A copy - Create new with Oracle 11 DG does not work )
Best Regards
mseberg
Edited by: mseberg on Mar 14, 2012 5:10 PM

What would happened when one RAC node's public NIC down ?

Dear all,
There's a two-node RAC on my office. As my observation, when one RAC node's public NIC is down, the "crs_stat -t" command wouldn't show any resource is OFFLINE. So at this moment if i use sqlplus to connect to my RAC db, it still will choose node1 or node2 instance randomly right? And if i've been assigned to the node's instance whose public NIC is down, my sqlplus would failed to connect to db?
So, can we say that Oracle RAC can't supply HA function if one node's public NIC is down? Or is there just any other solution to solve this issue? Any suggestion would be appreciated.

Public node is down means that only one node would be able to take the load i.e. in your case it will be Node 2. It may happen that CRS is unable to record the status - in such case there may be failed connections to one node.

RAC Node down and ORA-12514

Similar Messages

Maybe you are looking for