Query on setting of ORACLE_HOME in RAC node and more errors
i m following oracle docs for install Oracle 10g R2 RAC on RHEL 5.
i am able to get through most of the part but one minor thing is still bugging me.
Enter commands similar to the following to set the ORACLE_BASE and
ORACLE_HOME environment variable in preparation for the Oracle Clusterware
installation:
■ Bourne, Bash, or Korn shell:
$ ORACLE_BASE=/u01/app/oracle
$ ORACLE_HOME=/u01/crs/oracle/product/10/app
$ export ORACLE_BASE
$ export ORACLE_HOMEthe above extract from the docs (B14203-09) says ORACLE_HOME should be set to the dir where clusterware is installed.
my query is do we change this ORACLE_HOME to '/u01/app/oracle/product/10/db_1' when we install the oracle database after we finish installing the clusterware.
Further, permissions for the /u01/crs directory should be root:oinstall and 775 during the installation of the clusterware and changed to 644 after installation of clusterware. is that correct? is it 644 or 640? does it need read access for every one?
Edited by: iinfi on Jul 5, 2009 1:42 AM
Edited by: iinfi on Jul 5, 2009 12:20 PM
i initially thought not setting the oracle_home to the crs directory was the cause of the error message
[root@node1 ~]# /ora/app/oracle/product/10.2.0/crs/root.sh
WARNING: directory '/ora/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/ora/app/oracle/product' is not owned by root
WARNING: directory '/ora/app/oracle' is not owned by root
WARNING: directory '/ora/app' is not owned by root
WARNING: directory '/ora' is not owned by root
Checking to see if Oracle CRS stack is already configured
Setting the permissions on OCR backup directory
Setting up NS directories
Failed to upgrade Oracle Cluster Registry configurationthe crs directory is owned by root:oinstall with permissions set to 775. CRS and voting disks are raw devices set on my RHEL 5.3 with openfiler as storage (iSCSI targets)
[root@node1 node1]# cat /ora/app/oracle/product/10.2.0/crs/log/node1/client/ocrconfig_9680.log
Oracle Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 Oracle. All rights reserved.
2009-07-05 02:03:01.250: [ OCRCONF][14166224]ocrconfig starts...
2009-07-05 02:03:01.253: [ OCRCONF][14166224]Upgrading OCR data
2009-07-05 02:03:01.300: [ OCRCONF][14166224]OCR already in current version.
2009-07-05 02:03:01.363: [ OCRCONF][14166224]Failed to call clsssinit (21)
2009-07-05 02:03:01.363: [ OCRCONF][14166224]Failed to make a backup copy of OCR
2009-07-05 02:03:01.363: [ OCRCONF][14166224]Exiting [status=failed]...can someone please throw some light as to what i am missing here.
thanks
Similar Messages
-
Hi,
I have a 10.2.0.3 RAC running on RHELv4. What is a node and disk ping in terms of Oracle RAC? I assume the node ping refers to the hangcheck-timer, or is this wrong? The only disk ping that I recall was in 9i but not heard of this in 10g unless it's something else.
Also, where would I configure these parameters?
Thanks.Try ML Note 294430.1: "CSS Timeout Computation in RAC 10g (10g Release 1 and 10g Release 2)" for example.
Thanks. -
Does Maintenance Wizard allow to register RAC node and if yes, then how?
Maintenance Wizard is not allowing to register additional nodes for RAC. This cause manual run for other nodes. Is there any way we can setup multiple nodes?
Example, we have two CM/Admin nodes and RAC database. We are using Maintenance Wizard version 2.19.
ThanksEdit the following script to sort out bug 11699526.
$EOF_HOME/scripts/COMMON/eof1122e.sh
Line 131 reads "(CONNECT_DATA=(SID=${DB_SID})" change it to read "(CONNECT_DATA=(SERVICE_NAME=${DB_SID})".
Enter the database service name when configuring the DB nodes.
You will still need to do the OS related operations manually on node 2.
Robert -
I am writing a query to determine some information from a work breakdown numbering structure (single column) and it is simple enough to describe, just not sure how to do it in code. I am trying to determine for the parent items, how many children items
that parent has and how many children folders are under that parent (think of something like a folder/file structure). The numbering structure looks like this:
1
1.1
1.2
1.2.1
1.2.2
1.2.3
1.2.3.1
1.2.3.2
1.2.3.3
1.2.4
1.2.5
1.2.5.1
1.2.5.2
1.3
I would like my query to return this information:
WBS ChildFolders ChildLineItems
1.1 0 0
1.2 2 8
1.3 0 0
Hope that makes sense!I almost have this working in the following query in at least I figure out a way to tell how which ones were the folders and which were the line items with this query (sorry I have different WBS numbers than my example above:
declare @wbs table (wbs varchar(max))
insert @wbs select
'1.1' as wbs union all select
'1.2' union all select
'1.2.1' union all select
'1.2.2' union all select
'1.2.3' union all select
'1.3' union all select
'1.4' union all select
'1.4.1' union all select
'1.4.1.1' union all select
'1.4.1.2' union all select
'1.4.2' union all select
'1.5.1'
select
x.wbs
, CASE count(x.wbs) WHEN 1 THEN 'LineItem' ELSE 'Folder/Has Children' END as Description
from
@wbs x
,(select
from @wbs w) as data
where
data.wbs like x.wbs + '%'
group by
x.wbs
This give me these results:
WBS Description
1.1 LineItem
1.2 Folder/Has Children
1.2.1 LineItem
1.2.2 LineItem
1.2.3 LineItem
1.3 LineItem
1.4 Folder/Has Children
1.4.1 Folder/Has Children
1.4.1.1 LineItem
1.4.1.2 LineItem
1.4.2 LineItem
1.5.1 LineItem
I would just like to add the columns with the counts of ChildFolders and ChildLineItems like my example above.
For example the results for 1.4 would look like this:
WBS Description
ChildFolders ChildLineItems
1.4 Folder/Has Children
1 3 -
I bought a new 4th gen ipod shuffle and tried to sdet it up last night. I installed the latest itunes software and started to sync my ipod, but error boxes kept popping up saying ipod not found cannot sync. It was showing in itunes, then it would disappear, then it would come back. I am not very good with computers so I don't know what to do. Can someone help?
Do you have an iCloud backup? If you backed up your old phone to iCloud you can go to Settings>General>Reset>Erast All Content & Settings (which will erase your phone), then go back through the setup screens again and when prompted choose Restore From iCloud Backup. However your backup won't contain everything in your iTunes library (such as your music) so you'll need to be able to sync with your computer unless you purchased everything from iTunes and want to redownload it to your phone.
-
Rac node failed how do you bring it back up?
Example: If there are 3 RAC nodes and one becomes unavailable/fails, how do you bring it back up?
There are typically two basic reasons why a RAC node will go down.
A cluster issue, causing the node to be evicted. This usually means the node lost access to the cluster storage (e.g. no access to voting disks), or lost access to the Interconnect.
An o/s issue causing the node to fail. E.g. a kernel panic due to a page swap and memory not syncing, or a soft CPU lockup, etc.
You need to determine why it went down, and determine what is needed to enable it to join the successfully cluster again. -
Jtree Select node and change leafs icon problem
Hi All,
i create a tree and implement a TreeSelectionListener:
my mission is whenever i select a node i need to change the icon of this node (for now.later i will have to find if it have childrens).
import java.awt.Color;
import java.awt.Component;
import java.util.Enumeration;
import java.util.NoSuchElementException;
import java.util.Vector;
import javax.swing.ImageIcon;
import javax.swing.JTree;
import javax.swing.event.TreeExpansionEvent;
import javax.swing.event.TreeExpansionListener;
import javax.swing.event.TreeSelectionEvent;
import javax.swing.event.TreeSelectionListener;
import javax.swing.tree.DefaultMutableTreeNode;
import javax.swing.tree.DefaultTreeCellRenderer;
import javax.swing.tree.TreeModel;
import javax.swing.tree.TreeNode;
import javax.swing.tree.TreePath;
public class TreeView{
DefaultMutableTreeNode top;
JTree tree ;
Color frameColor;
public static ImageIcon NoTSelIcon;
public static ImageIcon SelIcon;
public static String[] name= new String[8];
public TreeView(Color BackColor) {
// TODO Auto-generated constructor stub
top = new DefaultMutableTreeNode("Diagnostics");
this.frameColor=BackColor;
SelIcon = createImageIcon("../Resource/Images/Select.gif");
if (SelIcon == null)
System.err.println("Tutorial icon missing; using default.");
NoTSelIcon = createImageIcon("../Resource/Images/NotSelc.gif");
if (NoTSelIcon == null)
System.err.println("Tutorial icon missing; using default.");
public Component createTreeComponents(){
//Create the nodes.
createNodes(top);
//Create a tree that allows one selection at a time.
tree = new JTree(top);
//TREE LISTENERS
//Treeselction listener
Handler hObject = new Handler();
tree.addTreeSelectionListener(hObject);
//Tree expand/collapse listener
HandlerExpansionListener hObjectExpan = new HandlerExpansionListener();
tree.addTreeExpansionListener(hObjectExpan);
// tree.getSelectionModel().setSelectionMode(TreeSelectionModel.DISCONTIGUOUS_TREE_SELECTION);
//set tree background
tree.setBackground(frameColor);
tree.setCellRenderer(new OverrideTreeCellRenderer(frameColor,SelIcon,NoTSelIcon));
return tree;
private void createNodes(DefaultMutableTreeNode top) {
DefaultMutableTreeNode category = null;
DefaultMutableTreeNode SubCategory = null;
DefaultMutableTreeNode SubCategoryBasee = null;
DefaultMutableTreeNode SubSubCategoryBasee = null;
category = new DefaultMutableTreeNode("Dfe");
top.add(category);
//Sub test visible
SubCategory = new DefaultMutableTreeNode("Test Visible");
category.add(SubCategory);
SubCategory.add(new DefaultMutableTreeNode("Son 1"));
SubCategory.add(new DefaultMutableTreeNode("Son 2"));
SubSubCategoryBasee = new DefaultMutableTreeNode("Test Base");
SubSubCategoryBasee.add(new DefaultMutableTreeNode("Grandson 1"));
SubSubCategoryBasee.add(new DefaultMutableTreeNode("Grandson 2"));
SubCategory.add(SubSubCategoryBasee);
class Handler implements TreeSelectionListener {
public void valueChanged(TreeSelectionEvent arg0) {
// TODO Auto-generated method stub
System.out.println("treeSelect event ");
TreePath trph;
trph=arg0.getNewLeadSelectionPath();
int count=trph.getPathCount();
DefaultMutableTreeNode Selnode = (DefaultMutableTreeNode)tree.getLastSelectedPathComponent();
String Name = (String)Selnode.getUserObject();
setSelected(Selnode,true);
int number_ofnodes=getNodeCountBelow((TreeModel)tree.getModel() , Selnode, false);
System.out.println("The Number of nodes under "+Name+"="+number_ofnodes);
tree.setCellRenderer(new IconRenderer(SelIcon,NoTSelIcon,frameColor));
class HandlerExpansionListener implements TreeExpansionListener {
public void valueChanged(TreeSelectionEvent arg0) {
// TODO Auto-generated method stub
DefaultMutableTreeNode node = (DefaultMutableTreeNode) tree.getLastSelectedPathComponent();
if (node == null) return;
} // The inner class
public void treeCollapsed(TreeExpansionEvent arg0) {
// TODO Auto-generated method stub
System.out.println("treeCollapsed event ");
public void treeExpanded(TreeExpansionEvent arg0) {
// TODO Auto-generated method stub
System.out.println("treeExpanded event ");
/** Returns an ImageIcon, or null if the path was invalid. */
protected static ImageIcon createImageIcon(String path) {
//ImageIcon imcon= new ImageIcon(path);
//return imcon;
java.net.URL imgURL = TreeView.class.getResource(path);
if (imgURL != null) {
return new ImageIcon(imgURL);
} else {
System.err.println("Couldn't find file: " + path);
return null;
DefaultMutableTreeNode newnode;
public void setSelected(DefaultMutableTreeNode Selnode ,boolean isSelected)
Enumeration Enchilds=Selnode.children();//ENUMRATE ALL CHILDS FOR THIS NODE
if (Enchilds != null)
while (Enchilds.hasMoreElements())
newnode=(DefaultMutableTreeNode)Enchilds.nextElement();
String NameSel = (String)newnode.getUserObject();
setSelected(newnode,isSelected);
//GETTING THE TREE DEPTH
public int getNodeCountBelow(TreeModel model, Object node, boolean includeInitialNode)
int n = includeInitialNode ? 1 : 0;
for (int i = 0; i < model.getChildCount(node); i ++)
n += getNodeCountBelow(model, model.getChild(node, i), true);
return n;
import java.awt.Color;
import java.awt.Component;
import java.util.Enumeration;
import java.util.NoSuchElementException;
import javax.swing.Icon;
import javax.swing.JTree;
import javax.swing.tree.DefaultMutableTreeNode;
import javax.swing.tree.DefaultTreeCellRenderer;
public class IconRenderer extends DefaultTreeCellRenderer {
private static final long serialVersionUID = 1L;
Icon SelectedIcon;
Icon NotSelectedIcon;
Color BackgroundColor;
boolean Selected=false;
boolean Leaf=false;
boolean IsItaChild=false;
DefaultMutableTreeNode SelctedNode=null;
public IconRenderer(Icon SelIcon,Icon NoTSelIcon,Color Bacground) {
SelectedIcon = SelIcon;
NotSelectedIcon = NoTSelIcon;
BackgroundColor=Bacground;
setBackgroundNonSelectionColor(BackgroundColor);
public Component getTreeCellRendererComponent(JTree tree,Object value,boolean sel,boolean expanded,
boolean leaf,int row,boolean hasFocus)
super.getTreeCellRendererComponent(tree, value, sel,expanded, leaf, row,hasFocus);
Selected=sel;
Leaf=leaf;
DefaultMutableTreeNode node = (DefaultMutableTreeNode) value;
String s2 = (String)node.getUserObject();
return this;
}my problem is :
when i select a node the the method "getTreeCellRendererComponent"
start to run on the entire tree from buttom to top and than from top to buttom.
for me it waste of time because if has say 100 nodes it wont botthers me.
but i have 20000 nodes and more its take a time.
and for all this nodes i have to make compares.
is there a way to force the DefaultTreeCellRenderer to not run the entire tree???
ThanksYou need to make sure that your TreeModel interprets your group nodes to be non-leaf nodes (one of the methods in the TreeModel interface is called isLeaf). If you are using a DefaultTreeModel with DefaultMutableTreeNode objects, you can use the askAllowsChildren property of DefaultTreeModel and the allowsChildren property of DefaultMutableTreeNode to control this. See the API for more details:
http://java.sun.com/j2se/1.4.2/docs/api/javax/swing/tree/DefaultTreeModel.html
http://java.sun.com/j2se/1.4.2/docs/api/javax/swing/tree/DefaultMutableTreeNode.html -
Hello everyone,
I have met an error,that is our RAC node auto restart with below messages.
#/u01/app/oracle/diag/rdbms/odsdb/odsdb1/trace/alert_odsdb1.log
Fri Jun 07 12:23:42 2013
Thread 1 cannot allocate new log, sequence 58363
Checkpoint not complete
Current log# 2 seq# 58362 mem# 0: +DATA/odsdb/onlinelog/group_2.265.812288839
Current log# 2 seq# 58362 mem# 1: +DATA/odsdb/onlinelog/group_2.266.812288839
Fri Jun 07 12:23:42 2013
NOTE: ASMB terminating
Errors in file /u01/app/oracle/diag/rdbms/odsdb/odsdb1/trace/odsdb1_asmb_32641.trc:
ORA-15064: ? ASM ??????
ORA-03113: ?????????
?? ID:
?? ID: 2047 ???: 5
Errors in file /u01/app/oracle/diag/rdbms/odsdb/odsdb1/trace/odsdb1_asmb_32641.trc:
ORA-15064: ? ASM ??????
ORA-03113: ?????????
?? ID:
?? ID: 2047 ???: 5
ASMB (ospid: 32641): terminating the instance due to error 15064
Fri Jun 07 12:23:44 2013
ORA-1092 : opitsk aborting process
Fri Jun 07 12:23:46 2013
ORA-1092 : opitsk aborting process
Instance terminated by ASMB, pid = 32641
Fri Jun 07 12:25:02 2013
Starting ORACLE instance (normal)
Fri Jun 07 12:25:23 2013
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
[name='eth1:1', type=1, ip=169.254.37.103, mac=00-26-55-eb-61-89, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
[name='eth0', type=1, ip=135.33.2.8, mac=00-26-55-eb-61-88, net=135.33.2.0/27, mask=255.255.255.224, use=public/1]
Public Interface 'eth0:1' configured from GPnP for use as a public interface.
[name='eth0:1', type=1, ip=135.33.2.13, mac=00-26-55-eb-61-88, net=135.33.2.0/27, mask=255.255.255.224, use=public/1]
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as /u01/app/oracle/product/11.2.0/dbhome_2/dbs/arch
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_2
System name: Linux
Node name: odsdb1
Release: 2.6.18-308.el5
Version: #1 SMP Fri Jan 27 17:17:51 EST 2012
Machine: x86_64
Using parameter settings in server-side pfile /u01/app/oracle/product/11.2.0/dbhome_2/dbs/initodsdb1.ora
System parameters with non-default values:
processes = 4500
sessions = 6784
event = ""
spfile = "+DATA/odsdb/spfileodsdb.ora"
nls_language = "SIMPLIFIED CHINESE"
nls_territory = "CHINA"
memory_target = 170G
control_files = "+DATA/odsdb/controlfile/current.262.812288837"
control_files = "+DATA/odsdb/controlfile/current.261.812288837"
db_block_size = 8192
compatible = "11.2.0.0.0"
db_files = 4096
cluster_database = TRUE
db_create_file_dest = "+DATA"
db_recovery_file_dest = ""
db_recovery_file_dest_size= 38820M
thread = 1
undo_tablespace = "UNDOTBS1"
instance_number = 1
remote_login_passwordfile= "EXCLUSIVE"
db_domain = ""
dispatchers = "(PROTOCOL=TCP) (SERVICE=odsdbXDB)"
remote_listener = "odsdb-cluster-scan:1521"
job_queue_processes = 1000
audit_file_dest = "/u01/app/oracle/admin/odsdb/adump"
audit_trail = "DB"
db_name = "odsdb"
open_cursors = 300
diagnostic_dest = "/u01/app/oracle"
Cluster communication is configured to use the following interface(s) for this instance
169.254.37.103
cluster interconnect IPC version:Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
Fri Jun 07 12:25:33 2013
PMON started with pid=2, OS id=22959
Fri Jun 07 12:25:33 2013
PSP0 started with pid=3, OS id=22962
Fri Jun 07 12:25:34 2013
VKTM started with pid=4, OS id=22971 at elevated priority
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Fri Jun 07 12:25:34 2013
GEN0 started with pid=5, OS id=22977
Fri Jun 07 12:25:34 2013
DIAG started with pid=6, OS id=22979
Fri Jun 07 12:25:35 2013
DBRM started with pid=7, OS id=22981
Fri Jun 07 12:25:35 2013
PING started with pid=8, OS id=22983
Fri Jun 07 12:25:35 2013
ACMS started with pid=9, OS id=22985
Fri Jun 07 12:25:35 2013
DIA0 started with pid=10, OS id=22987
Fri Jun 07 12:25:35 2013
LMON started with pid=11, OS id=22989
Fri Jun 07 12:25:35 2013
LMD0 started with pid=12, OS id=22991
* Load Monitor used for high load check
* New Low - High Load Threshold Range = [61440 - 81920]
Fri Jun 07 12:25:35 2013
LMS0 started with pid=13, OS id=22994 at elevated priority
Fri Jun 07 12:25:35 2013
LMS1 started with pid=14, OS id=22998 at elevated priority
Fri Jun 07 12:25:35 2013
LMS2 started with pid=15, OS id=23002 at elevated priority
Fri Jun 07 12:25:35 2013
LMS3 started with pid=16, OS id=23006 at elevated priority
Fri Jun 07 12:25:35 2013
RMS0 started with pid=17, OS id=23010
Fri Jun 07 12:25:35 2013
LMHB started with pid=18, OS id=23013
Fri Jun 07 12:25:35 2013
MMAN started with pid=19, OS id=23015
Fri Jun 07 12:25:35 2013
DBW0 started with pid=20, OS id=23017
Fri Jun 07 12:25:35 2013
DBW1 started with pid=21, OS id=23019
Fri Jun 07 12:25:35 2013
DBW2 started with pid=22, OS id=23022
Fri Jun 07 12:25:35 2013
DBW3 started with pid=23, OS id=23024
Fri Jun 07 12:25:35 2013
DBW4 started with pid=24, OS id=23026
Fri Jun 07 12:25:35 2013
DBW5 started with pid=25, OS id=23028
Fri Jun 07 12:25:35 2013
DBW6 started with pid=26, OS id=23031
Fri Jun 07 12:25:35 2013
DBW7 started with pid=27, OS id=23033
Fri Jun 07 12:25:35 2013
LGWR started with pid=28, OS id=23035
Fri Jun 07 12:25:35 2013
CKPT started with pid=29, OS id=23037
Fri Jun 07 12:25:35 2013
SMON started with pid=30, OS id=23039
Fri Jun 07 12:25:35 2013
RECO started with pid=31, OS id=23041
Fri Jun 07 12:25:35 2013
RBAL started with pid=32, OS id=23043
Fri Jun 07 12:25:35 2013
ASMB started with pid=33, OS id=23045
Fri Jun 07 12:25:35 2013
MMON started with pid=34, OS id=23048
Fri Jun 07 12:25:35 2013
MMNL started with pid=35, OS id=23052
Fri Jun 07 12:25:35 2013
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
NOTE: initiating MARK startup
starting up 1 shared server(s) ...
Starting background process MARK
Fri Jun 07 12:25:35 2013
MARK started with pid=37, OS id=23056
NOTE: MARK has subscribed
lmon registered with NM - instance number 1 (internal mem no 0)
Reconfiguration started (old inc 0, new inc 119)
List of instances:
1 2 (myinst: 1)
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
* domain 0 valid according to instance 2
* domain 0 valid = 1 according to instance 2
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
LMS 3: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
LMS 1: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
LMS 2: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Submitted all GCS remote-cache requests
Fix write in gcs resources
Reconfiguration started (old inc 119, new inc 121)
List of instances:
1 2 (myinst: 1)
Nested reconfiguration detected.
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
LMS 3: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
LMS 2: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
LMS 1: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Fri Jun 07 12:25:45 2013
Submitted all GCS remote-cache requests
Fri Jun 07 12:26:08 2013
Fix write in gcs resources
Reconfiguration complete
Fri Jun 07 12:26:10 2013
LCK0 started with pid=40, OS id=23632
Fri Jun 07 12:26:10 2013
Starting background process RSMN
Fri Jun 07 12:26:10 2013
RSMN started with pid=41, OS id=23646
ORACLE_BASE not set in environment. It is recommended
that ORACLE_BASE be set in the environment
Reusing ORACLE_BASE from an earlier startup = /u01/app/oracle
Fri Jun 07 12:26:11 2013
ALTER SYSTEM SET local_listener=' (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=135.33.2.13)(PORT=1521))))' SCOPE=MEMORY SID='odsdb1';
ALTER DATABASE MOUNT /* db agent *//* {1:9971:2} */
Fri Jun 07 12:26:11 2013
NOTE: Loaded library: System
Fri Jun 07 12:26:11 2013
SUCCESS: diskgroup DATA was mounted
Fri Jun 07 12:26:11 2013
NOTE: dependency between database odsdb and diskgroup resource ora.DATA.dg is established
Fri Jun 07 12:26:16 2013
Successful mount of redo thread 1, with mount id 3452000551
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Lost write protection disabled
Completed: ALTER DATABASE MOUNT /* db agent *//* {1:9971:2} */
ALTER DATABASE OPEN /* db agent *//* {1:9971:2} */
Picked broadcast on commit scheme to generate SCNs
Thread 1 advanced to log sequence 58364 (thread open)
Thread 1 opened at log sequence 58364
Current log# 2 seq# 58364 mem# 0: +DATA/odsdb/onlinelog/group_2.265.812288839
Current log# 2 seq# 58364 mem# 1: +DATA/odsdb/onlinelog/group_2.266.812288839
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Fri Jun 07 12:26:21 2013
SMON: enabling cache recovery
Fri Jun 07 12:26:23 2013
minact-scn: Inst 1 is a slave inc#:121 mmon proc-id:23048 status:0x2
minact-scn status: grec-scn:0x0000.00000000 gmin-scn:0x0000.00000000 gcalc-scn:0x0000.00000000
Fri Jun 07 12:26:34 2013
[23651] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:2061372614 end:2061384964 diff:12350 (123 seconds)
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
Fri Jun 07 12:26:34 2013
SMON: enabling tx recovery
Database Characterset is ZHS16GBK
No Resource Manager plan active
Starting background process GTX0
Fri Jun 07 12:26:35 2013
GTX0 started with pid=45, OS id=23931
Starting background process RCBG
Fri Jun 07 12:26:35 2013
RCBG started with pid=46, OS id=23933
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
Fri Jun 07 12:26:35 2013
QMNC started with pid=48, OS id=23940
Completed: ALTER DATABASE OPEN /* db agent *//* {1:9971:2} */
Fri Jun 07 12:26:38 2013
Starting background process CJQ0
Fri Jun 07 12:26:38 2013
CJQ0 started with pid=55, OS id=23977
Fri Jun 07 12:27:56 2013
Thread 1 advanced to log sequence 58365 (LGWR switch)
Current log# 1 seq# 58365 mem# 0: +DATA/odsdb/onlinelog/group_1.263.812288839
Current log# 1 seq# 58365 mem# 1: +DATA/odsdb/onlinelog/group_1.264.812288839
Fri Jun 07 12:28:18 2013
Starting background process SMCO
Fri Jun 07 12:28:18 2013
SMCO started with pid=70, OS id=25166
Fri Jun 07 12:29:01 2013
Thread 1 cannot allocate new log, sequence 58366
Trace file /u01/app/oracle/diag/rdbms/odsdb/odsdb1/trace/odsdb1_asmb_32641.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_2
System name: Linux
Node name: odsdb1
Release: 2.6.18-308.el5
Version: #1 SMP Fri Jan 27 17:17:51 EST 2012
Machine: x86_64
Instance name: odsdb1
Redo thread mounted by this instance: 0 <none>
Oracle process number: 33
Unix process pid: 32641, image: oracle@odsdb1 (ASMB)
*** 2013-05-14 15:37:08.705
*** SESSION ID:(3499.1) 2013-05-14 15:37:08.705
*** CLIENT ID:() 2013-05-14 15:37:08.705
*** SERVICE NAME:() 2013-05-14 15:37:08.705
*** MODULE NAME:() 2013-05-14 15:37:08.705
*** ACTION NAME:() 2013-05-14 15:37:08.705
NOTE: initiating MARK startup
*** 2013-05-14 15:37:16.835
instance health monitoring reports instance shutting down
*** 2013-06-07 12:23:42.700
NOTE: ASMB terminating
ORA-15064: ? ASM ??????
ORA-03113: ?????????
?? ID:
?? ID: 2047 ???: 5
error 15064 detected in background process
ORA-15064: ? ASM ??????
ORA-03113: ?????????
?? ID:
?? ID: 2047 ???: 5
kjzduptcctx: Notifying DIAG for crash event
----- Abridged Call Stack Trace -----
ksedsts()+461<-kjzdssdmp()+267<-kjzduptcctx()+232<-kjzdicrshnfy()+53<-ksuitm()+1332<-ksbrdp()+3344<-opirip()+623<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+252<-main()+201<-__libc_start_main()+244<-_start()+36
----- End of Abridged Call Stack Trace -----
*** 2013-06-07 12:23:42.783
ASMB (ospid: 32641): terminating the instance due to error 15064
/u01/app/grid/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log
NOTE: ASMB process exiting, either shutdown is in progress
NOTE: or foreground connected to ASMB was killed.
Fri Jun 07 12:23:42 2013
NOTE: client exited [14808]
Fri Jun 07 12:23:44 2013
Received an instance abort message from instance 2
Please check instance 2 alert and LMON trace files for detail.
Fri Jun 07 12:23:44 2013
Received an instance abort message from instance 2
Please check instance 2 alert and LMON trace files for detail.
LMD0 (ospid: 31201): terminating the instance due to error 481
Instance terminated by LMD0, pid = 31201
Fri Jun 07 12:24:30 2013
* instance_number obtained from CSS = 1, checking for the existence of node 0...
* node 0 does not exist. instance_number = 1
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
[name='eth1:1', type=1, ip=169.254.37.103, mac=00-26-55-eb-61-89, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
[name='eth0', type=1, ip=135.33.2.8, mac=00-26-55-eb-61-88, net=135.33.2.0/27, mask=255.255.255.224, use=public/1]
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as /u01/app/11.2.0.2/grid/dbs/arch
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
[grid@odsdb1 cssd]$ file core.30481
core.30481: ELF 64-bit LSB core file AMD x86-64, version 1 (SYSV), SVR4-style, from 'ocssd.bin'
[grid@odsdb1 cssd]$ gdb
gdb gdbserver gdbtui
[grid@odsdb1 cssd]$ gdb ocssd.bin core.30481
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-42.el5)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /u01/app/11.2.0.2/grid/bin/ocssd.bin...(no debugging symbols found)...done.
[New Thread 30486]
[New Thread 30530]
[New Thread 30526]
[New Thread 30525]
[New Thread 30523]
[New Thread 30522]
[New Thread 30521]
[New Thread 30520]
[New Thread 30519]
[New Thread 30504]
[New Thread 30503]
[New Thread 30495]
[New Thread 30485]
[New Thread 30484]
[New Thread 30483]
[New Thread 30481]
Reading symbols from /u01/app/11.2.0.2/grid/lib/libhasgen11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libhasgen11.so
Reading symbols from /u01/app/11.2.0.2/grid/lib/libocr11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libocr11.so
Reading symbols from /u01/app/11.2.0.2/grid/lib/libocrb11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libocrb11.so
Reading symbols from /u01/app/11.2.0.2/grid/lib/libocrutl11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libocrutl11.so
Reading symbols from /u01/app/11.2.0.2/grid/lib/libclntsh.so.11.1...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libclntsh.so.11.1
Reading symbols from /u01/app/11.2.0.2/grid/lib/libskgxn2.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libskgxn2.so
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnsl.so.1
Reading symbols from /u01/app/11.2.0.2/grid/lib/libasmclntsh11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libasmclntsh11.so
Reading symbols from /u01/app/11.2.0.2/grid/lib/libcell11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libcell11.so
Reading symbols from /u01/app/11.2.0.2/grid/lib/libskgxp11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libskgxp11.so
Reading symbols from /u01/app/11.2.0.2/grid/lib/libnnz11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libnnz11.so
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /usr/lib64/libaio.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libaio.so.1
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /u01/app/11.2.0.2/grid/lib/libnque11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/11.2.0.2/grid/lib/libnque11.so
Reading symbols from /opt/oracle/extapi/64/asm/orcl/1/libasm.so...(no debugging symbols found)...done.
Loaded symbols for /opt/oracle/extapi/64/asm/orcl/1/libasm.so
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff505fd000
Core was generated by `/u01/app/11.2.0.2/grid/bin/ocssd.bin '.
Program terminated with signal 6, Aborted.
#0 0x000000369ea30265 in raise () from /lib64/libc.so.6
(gdb) where
#0 0x000000369ea30265 in raise () from /lib64/libc.so.6
#1 0x000000369ea31d10 in abort () from /lib64/libc.so.6
#2 0x00002afc67f9aeda in scls_abort (flags=0) at scls.c:7088
#3 0x000000000040babd in clssscExit (thrd=0x10d325a0, status=clssscreasonSHUTNORM) at clsssc.c:2155
#4 0x0000000000446221 in clssgmClientShutdown (thrd=0x10d325a0, cmInfo=0x10b40090) at clssgmc.c:6415
#5 0x0000000000436707 in clssgmProcClientReqs (thrd=0x10d325a0, clctx=0x10b40630) at clssgmc.c:704
#6 0x0000000000436405 in clssgmclientlsnr (thrd=0x10d325a0) at clssgmc.c:644
#7 0x000000000040ac2f in clssscthrdmain (thrd=0x10d325a0) at clsssc.c:1716
#8 0x000000369fa0677d in start_thread () from /lib64/libpthread.so.0
#9 0x000000369ead49ad in clone () from /lib64/libc.so.6
(gdb)
2013-06-07 12:19:37.377: [ CSSD][1085888832]clssscSelect: cookie accept request 0x10b40630
2013-06-07 12:19:37.377: [ CSSD][1085888832]clssgmAllocProc: (0x2aaab0133ea0) allocated
2013-06-07 12:19:37.379: [ CSSD][1085888832]clssgmClientConnectMsg: properties of cmProc 0x2aaab0133ea0 - 1,2,3,4,5
2013-06-07 12:19:37.379: [ CSSD][1085888832]clssgmClientConnectMsg: Connect from con(0x6ae44fa) proc(0x2aaab0133ea0) pid(14139/14139) version 11:2:1:4, properties: 1,2,3,4,5
2013-06-07 12:19:37.379: [ CSSD][1085888832]clssgmClientConnectMsg: msg flags 0x0000
2013-06-07 12:19:37.384: [ CSSD][1085888832]clssscSelect: cookie accept request 0x2aaab0133ea0
2013-06-07 12:19:37.384: [ CSSD][1085888832]clssscevtypSHRCON: getting client with cmproc 0x2aaab0133ea0
2013-06-07 12:19:37.384: [ CSSD][1085888832]clssgmRegisterClient: proc(69/0x2aaab0133ea0), client(1/0x2aaab010c5c0)
2013-06-07 12:19:37.385: [ CSSD][1085888832]clssgmRegisterShared: grp DBODSDB, mbr 0, type 1
2013-06-07 12:19:37.385: [ CSSD][1085888832]clssgmQueueShare: (0x2aaab0085790) target global grock DBODSDB member 0 type 1 queued from client (0x2aaab010c5c0), global grock DBODSDB, refcount 23
2013-06-07 12:19:37.385: [ CSSD][1085888832]clssgmRegisterShared: global grock DBODSDB member 0 share type 1, refcount 23
2013-06-07 12:19:37.391: [ CSSD][1085888832]clssscSelect: cookie accept request 0x2aaab0133ea0
2013-06-07 12:19:37.391: [ CSSD][1085888832]clssscevtypSHRCON: getting client with cmproc 0x2aaab0133ea0
2013-06-07 12:19:37.391: [ CSSD][1085888832]clssgmRegisterClient: proc(69/0x2aaab0133ea0), client(2/0x2aaab0061f10)
what is the problem
Edited by: 徐振富 on 2013-6-7 下午6:38
Edited by: 徐振富 on 2013-6-7 下午6:45is your ASM instance up?
If not, trying bring up ASM instance up just by itself and see if it throws any error?
Post status of crsctl status cluster -all -
Is it recommended to support multiple DBs and Instances on same RAC nodes
Customer wants to install multiple databases on the same set of 3-nodes essentially making all these database 3-node rac databases on Sun fire servers.
Is this a recommended approach? What are the pitfalls?
Can this be done effectively without using local containers (which are not supported for RAC)?The Customer would save significantly on Licencing if he does not have to add Processors into the three nodes.
Processor Based Licences don't count the number of databases and instances, only the number of CPUs (Processors).
Setting up seperate Clusters requires that many Processor Licences.
However, using multiple Standard Edition RAC installations (Linux and ASM) would have been another option.
I have run multiple databases on a single Cluster in 9iRAC. I am sure that there are customers running multiple databases in a single Cluster in 10gRAC. You'd have only one CRS and one ASM Home in each node but it may be possible to have separate Database ORACLE_HOMEs in each node as well. -
Multiple Standby Databases on same RAC nodes.
We have a 3 node Oracle 10gR2 RAC production environment on site A and a 3 node Oracle 10g RAC standby environment on site B. Both use like HW and OS - HP BL45p with RHEL AS 4.x.
Can we have heterogeneous standby databases(logical and physical) running on the same RAC nodes?
Can the 2 apply processes (MRP-manageed recovery process & LSP-logical standby process) coexists on the same set of nodes in a cluster at the same time. Are there any conflicts or limitations?
Is there any documentation that supports this?Would Active Data Guard give you the best of both worlds?
The caveat might be your SID_LIST_LISTENER setup.
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0)
(PROGRAM = extproc)
(SID_DESC =
(global_dbname = <database1>_DGMGRL.yourdomain)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0)
(sid_name = <database1>)
(SID_DESC =
(global_dbname = <database2>_DGMGRL.yourdomain)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0)
(sid_name = <database2>)
I have a server with ten standbys on it currently and no issues.
As long as the version is the same you should be good. -
Issues while Add / Delete RAC Node in Oracle 10g R2
Hi,
I have an requirement to add a New Node in the existing 2 Node RAC at Production, where 1 Node is Active & other one is passive due to licence issue & cannot keep both the nodes as active. Due to performance issues (Memory , CPU Cores ..etc) we are adding another new node.
Right now we are planning to add a 3rd database node making the new node as active and current active one as passive which is a swap & later on after final observation delete and decommission the current passive node.
This activity is checked at the Dev database with the same infrastructure (OS + Memory ..etc) but want to check what is the best approach (or) challenges we face during the RAC Node Addition / Deletion
RAC DB Version : 10.2.o.4
OS Version : RHEL 5.8
(1) Is the approach is right one , First Adding the node & later on delete
(2) If the approach is the correct , what would be the behavious of the 3rd node in means of active (or) passive
(3) We have taken RMAN backup , OS backup , CRS , ORACLE_HOME , ASM_Home backup , OCR & VD.
(4) Could you please give detail steps for adding / deleting node in 10g R2.
(5) Are they any known bugs to us with the DB release (or) OS while performing this activity.
Since this is a production machine we want to more proactive . Please correct or add any thing i am missing out ...
With Thanks,
RakeshHello Rakesh,
Please follow the following steps.
Node Addition Steps
1. Install and configure OS and hardware for new node.
2. Add Oracle Clusterware to the new node.
3. Configure ONS for the new node.
4. Add ASM home to the new node.
5. Add Databse home to the new node.
6. Add a listener to the new node.
7. Add ASM instance to the New Node.
8. Add a database instance to the new node.
Details of steps
1. run cluvfy to verify whether New node is ready for addition or not.
$ cluvfy stage -pre crsinst -n node2
2. from node1, execute
$/u01/app/crs11g/oui/bin/addNode.sh
3. Specify node2 vip address and follow instructions.
4. In the last of installtion it may through an wornig and will ask to click on YES. click on YES
5. from node1,
/u01/app/crs11g/bin/racgons add_config node2:6200
6. from Node1,set ORACLE_HOME=ASM_HOME and then execute addNode.sh from $ASM_HOME/oui/bin and Follow instrusctions.
7. From node1, set ORACLE_HOME=DB_HOME and then
/u01/app/oracle/product/11.1.0/db_1/oui/bin/addNode.sh
and Follow instructions.
8. from node2 start NETCA and configure listener for new node. While configuring Listener select the name of new node.
9. from node1 start dbca from ASM Home to configure ASM instance for new node.
10. Again from node1 start dbca from DB Home to add DB instance
Node deletion Steps
1. Delete the Database instance on the node to be deleted.
2. Clean up the ASM instance.
3. Remove the listener from the node to be deleted.
4. Remove the node from the database.
5. Remove the node from ASM.
6. Remove ONS configuration from the node to be deleted.
7. Remove the node from the clusterware
Details of Steps
1. Remove database Instance of node2
Dbca -> instance Management -> delete instance -> password for sys -> select node -> finish.
2. Stop asm for node2 from any nodes.
$srvctl stop asm –n node2
3. Remove asm for node2
$ srvctl remove asm -n node2
4. Remove Listener from Node2 using NETCA.
5. From Node2:
./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES=node2" -local
6. From Node2, start runinstaller from Oracle_DB_Home/oui/bin, and remove "DB_HOME"
$ ./runinstaller
On the WELCOME Screen -> Deinstall product -> Select dbhome name (OraDb10g_Home1) -> Remove
7. From Node1:
./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES=node1"
8. From Node2, set Oracle_Home to asm_1 and then fire:
./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES=node2" -local
9. From Node2, start OUI and deinstall ASM Home.
10. From Node1, Set ORACLE_HOME= /u01/app/oracle/product/11.1.0/asm_1
11. From Node1: from /u01/app/oracle/product/11.1.0/asm_1/oui/bin, start OUI
./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES=node1"
12. From Node2: as a root user (#) execute rootdelete.sh from /u01/app/crs11g/install
# /u01/app/crs11g/install/rootdelete.sh
13.From Node-1 first find out the node numbers
# /u01/app/crs11g/bin/olsnodes -n
output : node1 1
node2 2
14. From Node-1 as a root user (#):
# /u01/app/crs11g/install/rootdeletenode.sh node2[Node_Name] 2[node_no]
output:
CRS nodeapps are deleted successfully
clscfg: EXISTING configuration version 4 detected.
clscfg: version 4 is 11 Release 1.
Node deletion operation successful.
'node2' deleted successfully
15. From Node2 set ORACLE_HOME=CRS_HOME and then execute
$$ORACLE_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES=node2" CRS=TRUE -local
16. ./runInstaller and remove CRS_HOME
17. From Node-1:
$ /u01/app/crs11g/oui/bin/runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES=node1" CRS=TRUE
18. check node is deleted from ./crs_stat -t -
RAC node process using 25% physical memory
We have a QA server is non-RAC, and production is two-node RAC. We have a migration app that does an INSERT from SELECT over 2 instances. All of the machines have been in successful day-to-day use for several months...our only troublespot seems to be the migration app.
Today we started the app on the QA server and watched the oracle processes using top. They ran normally and finished without any problems.
The same app started on either of the RAC nodes produced process memory errors and died.
As the app ran, there was a process reading the data from the source instance and a process writing to the target instance. We confirmed this by querying the session data. It doesn't matter which of the nodes runs which target process...the result is the same.
The reading process(session) on the source instance seems to run normally. The write process on the target instance, however, begins slowly accumulating memory in about 16M chunks and holds on to them.
We saw this in the RES and in the MEM columns of top. The target process never released any memory, but slowly grabbed it until its
RES was 4GB and the %MEM was about 30%. The app then died with process memory error. This is reproducible over several runs.
( Per Metalink Note 567506.1, the recommended value for Linux 64-bit is 4294967295 ..we have that set. )
There are other oracle processes and instances running on both nodes which do not seem to be affected. The total number of processes on each machine is around 750..much lower than the nprocs ulimit of 63K.
These process are both oracle sessions spawned by the app.
I haven't seen any info on the web or Metalink that matches these symptoms, so I thought I'd try the experts.
Why would the write session continuously use up physical memory, but only on RAC nodes?
We are running RHEL5 on Dell Poweredge 2950 w 16K Physical mem. Version of 10g is R2.0.4.user12017889 wrote:
The write process on the target instance, however, begins slowly accumulating memory in about 16M chunks and holds on to them.Exactly what process is this? An Oracle server process? Dedicated or shared server?
We saw this in the RES and in the MEM columns of top. The target process never released any memory, but slowly grabbed it until its
RES was 4GB and the %MEM was about 30%. How does the writer process work? Does it use PL/SQL? Does it use bulk processing? How does it call the reader process? Or does the reader process call it? Is this over a database link.
The app then died with process memory error. This is reproducible over several runs.If this is an Oracle server process, then there should be an entry in the alert log of the instance that recorded the crash and includes the name of the trace file generated by the crash. -
3 NODE RAC win2003, emca gives error
Hi,
I have 3 node RAC, windows 2003 64 bit with ASM. Oracle version 10.2.0.4.0.
when i try to add emca i get error below.
I would appreciate any help that you can think of.
To add emca i used:
emca -config dbcontrol db -repos create - cluster -CLUSTER_NAME <cluster_name> -EM_NODE <DB Control's node name> -EM_SID_LIST <Agent sid list, separated by commas>
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setFlag
CONFIG: Flag '-config' set to true
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setFlag
CONFIG: Flag 'UPDATE_EMAIL' set to true
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setFlag
CONFIG: Flag 'dbcontrol' set to true
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setFlag
CONFIG: Flag 'db' set to true
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setFlag
CONFIG: Flag 'create' set to true
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setFlag
CONFIG: Flag 'drop' set to true
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: value: cluster
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: CLUSTER_NAME value: CRS
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: EM_NODE value: libro01
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: EM_SID_LIST value: opera1,opera2,opera3
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: db_unique_name value: opera
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: ORACLE_HOME value: D:\oracle\1020\DATABASE
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager getParam
CONFIG: No value was set for the parameter ORACLE_HOSTNAME.
Jan 20, 2010 5:03:35 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: EM_HOME value: D:\oracle\1020\DATABASE
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: SID value: opera1
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.ParamsManager getParam
CONFIG: No value was set for the parameter ORACLE_HOSTNAME.
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.util.ClusterUtil getLocalNode
CONFIG: Cluster.isCluster: true. Calling getLocalNode
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.util.ClusterUtil getLocalNode
CONFIG: isLocalNodeDone: true localNode: libro01
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.ParamsManager getParam
CONFIG: No value was set for the parameter ORACLE_HOSTNAME.
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.util.ClusterUtil isCRSHomeInstantaited
CONFIG: Value of CRS_HOME in D:\oracle\1020\DATABASE\sysman\config\emca.properties is D:/oracle/1020/CRS
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.util.DBControlUtil isDBConsoleConfigured
CONFIG: Sid: opera1 Host: Libro01 Node: libro01 OH: D:\oracle\1020\DATABASE isDBC: false
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.util.CentralAgentUtil isCentralAgentConfigured
CONFIG: Sid: opera1 Host: Libro01 Node: libro01 OH: D:\oracle\1020\DATABASE agentHome: null isCentral: false
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.util.CentralAgentUtil isCentralAgentConfigured
CONFIG: Sid: opera1 Host: Libro01 Node: libro01 OH: D:\oracle\1020\DATABASE agentHome: null isCentral: false
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.ParamsManager getParam
CONFIG: No value was set for the parameter SYS_PWD.
Jan 20, 2010 5:03:38 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: SQLEngine connecting with SID: opera1, oracleHome: D:\oracle\1020\DATABASE, and user:
Jan 20, 2010 5:03:39 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: SQLEngine created successfully and connected
Jan 20, 2010 5:03:39 PM oracle.sysman.emcp.ParamsManager setFlag
CONFIG: Flag 'asm_db' set to true
Jan 20, 2010 5:03:39 PM oracle.sysman.emcp.ParamsManager setFlag
CONFIG: Flag 'asm_db' set to true
Jan 20, 2010 5:03:39 PM oracle.sysman.emcp.ParamsManager getParam
CONFIG: No value was set for the parameter PORT.
Jan 20, 2010 5:03:43 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: PORT value: 1521
Jan 20, 2010 5:03:48 PM oracle.sysman.emcp.ParamsManager getParam
CONFIG: No value was set for the parameter SYS_PWD.
Jan 20, 2010 5:03:48 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: SQLEngine connecting with SID: opera1, oracleHome: D:\oracle\1020\DATABASE, and user: SYS
Jan 20, 2010 5:03:48 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: SQLEngine created successfully and connected
Jan 20, 2010 5:03:53 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: SQLEngine connecting with SID: opera1, oracleHome: D:\oracle\1020\DATABASE, and user: DBSNMP
Jan 20, 2010 5:03:53 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: SQLEngine created successfully and connected
Jan 20, 2010 5:03:59 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: EMAIL_ADDRESS value:
Jan 20, 2010 5:03:59 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: MAIL_SERVER_NAME value:
Jan 20, 2010 5:04:27 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: ASM_OH value: d:\oracle\1020\ASM
Jan 20, 2010 5:04:33 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: ASM_SID value: +ASM1
Jan 20, 2010 5:04:43 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: ASM_PORT value: 1521
Jan 20, 2010 5:04:49 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: ASM_USER_ROLE value: sydba
Jan 20, 2010 5:04:51 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: ASM_USER_NAME value: sys
Jan 20, 2010 5:04:56 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: SQLEngine connecting with SID: +ASM1, oracleHome: D:\oracle\1020\DATABASE, and user: sys
Jan 20, 2010 5:04:56 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: Re-initializing sql-engine. oldHome: D:\oracle\1020\DATABASE new home: D:\oracle\1020\DATABASE oldSid: opera1 new sid: +ASM1
Jan 20, 2010 5:04:57 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: SQLEngine created successfully and connected
Jan 20, 2010 5:04:57 PM oracle.sysman.emcp.ParamsManager getParam
CONFIG: No value was set for the parameter ORACLE_HOSTNAME.
Jan 20, 2010 5:04:57 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: CONFIRMATION ANSWER value:
Jan 20, 2010 5:05:12 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: CONFIRMATION ANSWER value: yes
Jan 20, 2010 5:05:12 PM oracle.sysman.emcp.ParamsManager getParam
CONFIG: No value was set for the parameter LOG_FILE.
Jan 20, 2010 5:05:12 PM oracle.sysman.emcp.ParamsManager getParam
CONFIG: No value was set for the parameter DB_UNIQUE_NAME.
Jan 20, 2010 5:05:12 PM oracle.sysman.emcp.ParamsManager getParam
CONFIG: No value was set for the parameter DB_UNIQUE_NAME.
Jan 20, 2010 5:05:12 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: SQLEngine connecting with SID: opera1, oracleHome: D:\oracle\1020\DATABASE, and user: SYS
Jan 20, 2010 5:05:12 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: Re-initializing sql-engine. oldHome: D:\oracle\1020\DATABASE new home: D:\oracle\1020\DATABASE oldSid: +ASM1 new sid: opera1
Jan 20, 2010 5:05:12 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngine
CONFIG: SQLEngine created successfully and connected
Jan 20, 2010 5:05:12 PM oracle.sysman.emcp.DatabaseChecks getDbUniqueName
CONFIG: No Database unique name available. Will try to retrieve it from DB itself
Jan 20, 2010 5:05:12 PM oracle.sysman.emcp.ParamsManager setParam
CONFIG: Setting param: DB_UNIQUE_NAME value: SP2-0640: Not connectedI was able to go bit further, now my error is below:
Connecting to database opera1 in home D:\oracle\1020\DATABASE with user "SYS" role "SYSDBA".
connected to database.
sql to execute declare repos_pwd varchar2(30); view_user varchar2(30); view_user_pwd varchar2(30); view_user_acc_status varchar2(30); begin repos_pwd := ?; sysman.mgmt_view_priv.get_view_user(view_user); select account_status into view_user_acc_status from sys.dba_users where upper(username) = upper ( view_user ); IF view_user_acc_status like '%LOCKED%' THEN execute immediate 'alter user ' || view_user || ' account unlock'; END IF; IF view_user_acc_status like '%EXPIRED%' THEN sysman.mgmt_view_priv.set_view_user_creds ( repos_pwd ); sysman.mgmt_view_priv.GET_VIEW_USER_CREDS ( view_user, view_user_pwd ); execute immediate 'alter user ' || view_user || ' identified by ' || view_user_pwd || ''; END IF; end;
noOfParams to bind: 1.
Jan 21, 2010 9:36:30 PM oracle.sysman.emcp.util.PlatformInterface executeCommand
CONFIG: Couldn't execute statement: ORA-06550: line 1, column 144:
PLS-00201: identifier 'SYSMAN.MGMT_VIEW_PRIV' must be declared
ORA-06550: line 1, column 144:
PL/SQL: Statement ignored
ORA-06550: line 1, column 470:
PLS-00201: identifier 'SYSMAN.MGMT_VIEW_PRIV' must be declared
ORA-06550: line 1, column 470:
PL/SQL: Statement ignored
ORA-06550: line 1, column 527:
PLS-00201: identifier 'SYSMAN.MGMT_VIEW_PRIV' must be declared
ORA-06550: line 1, column 527:
PL/SQL: Statement ignored (DBD ERROR: error possibly near <*> indicator at char 143 in 'declare repos_pwd varchar2(30); view_user varchar2(30); view_user_pwd varchar2(30); view_user_acc_status varchar2(30); begin repos_pwd := :p1; <*>sysman.mgmt_view_priv.get_view_user(view_user); select account_status into view_user_acc_status from sys.dba_users where upper(username) = upper ( view_user ); IF view_user_acc_status like '%LOCKED%' THEN execute immediate 'alter user ' || view_user || ' account unlock'; END IF; IF view_user_acc_status like '%EXPIRED%' THEN sysman.mgmt_view_priv.set_view_user_creds ( repos_pwd ); sysman.mgmt_view_priv.GET_VIEW_USER_CREDS ( view_user, view_user_pwd ); execute immediate 'alter user ' || view_user || ' identified by ' || view_user_pwd || ''; END IF; end;') at D:\oracle\1020\DATABASE\sysman\admin\scripts\emca\emcaDbUtil.pl line 108, <STDIN> line 2.
Jan 21, 2010 9:36:30 PM oracle.sysman.emcp.util.PlatformInterface executeCommand
CONFIG: Error executing CMD /C D:\oracle\1020\DATABASE\sysman\admin\scripts\emca\emcaDbUtil.bat "D:\oracle\1020\DATABASE\sysman\admin\scripts;D:\oracle\1020\DATABASE\bin;" D:/oracle/1020/DATABASE/perl/5.8.3/bin/MSWin32-x64-multi-thread\perl.exe D:\oracle\1020\DATABASE\sysman\admin\scripts\emca\emcaDbUtil.pl D:\oracle\1020\DATABASE opera1 SYS SYSDBA declare repos_pwd varchar2(30); view_user varchar2(30); view_user_pwd varchar2(30); view_user_acc_status varchar2(30); begin repos_pwd := ?; sysman.mgmt_view_priv.get_view_user(view_user); select account_status into view_user_acc_status from sys.dba_users where upper(username) = upper ( view_user ); IF view_user_acc_status like '%LOCKED%' THEN execute immediate 'alter user ' || view_user || ' account unlock'; END IF; IF view_user_acc_status like '%EXPIRED%' THEN sysman.mgmt_view_priv.set_view_user_creds ( repos_pwd ); sysman.mgmt_view_priv.GET_VIEW_USER_CREDS ( view_user, view_user_pwd ); execute immediate 'alter user ' || view_user || ' identified by ' || view_user_pwd || ''; END IF; end; 1 SYSMAN_PWD
Jan 21, 2010 9:36:30 PM oracle.sysman.emcp.EMReposConfig unlockMGMTAccount
CONFIG: Failed to unlock mgmt_view account
Jan 21, 2010 9:36:30 PM oracle.sysman.emcp.EMReposConfig invoke
SEVERE: Failed to unlock all EM-related accounts
Jan 21, 2010 9:36:30 PM oracle.sysman.emcp.EMConfig perform
SEVERE: Failed to unlock all EM-related accounts
Refer to the log file at D:\oracle\1020\DATABASE\cfgtoollogs\emca\opera\emca_2010-01-21_09-30-08-PM.log for more details.
Jan 21, 2010 9:36:30 PM oracle.sysman.emcp.EMConfig perform
CONFIG: Stack Trace:
oracle.sysman.emcp.exception.EMConfigException: Failed to unlock all EM-related accounts
at oracle.sysman.emcp.EMReposConfig.invoke(EMReposConfig.java:302)
at oracle.sysman.emcp.EMReposConfig.invoke(EMReposConfig.java:134)
at oracle.sysman.emcp.EMConfig.perform(EMConfig.java:171)
at oracle.sysman.emcp.EMConfigAssistant.invokeEMCA(EMConfigAssistant.java:486)
at oracle.sysman.emcp.EMConfigAssistant.performConfiguration(EMConfigAssistant.java:1142)
at oracle.sysman.emcp.EMConfigAssistant.statusMain(EMConfigAssistant.java:470)
at oracle.sysman.emcp.EMConfigAssistant.main(EMConfigAssistant.java:419) -
Oracle ORA_16000 when trying to add standby instance to existing rac node
I attempted to use dbca to add a new standby instance to an existing cluster. The cluster is 4 nodes, Linux RHEL 5.3 Oracle 11.1.0.7. Also using ASM, asmlib, ocfs2 and shared block devices.
ASM instances are up and functional on all nodes. current config appears to be running normally and correctly.
I have a 4 instance database running on the cluster. I also have 3 physical standby active data guard instances running on 3 of the nodes. I wanted to add a new ADG instance to the 4th node.
While running dbca I received ORA-00604 and ORA-16000.
The active data guard database was open (read only) and redo apply was on. I am using data guard broker as well, but not grid control.
Does anyone have a procedure for adding an instance in this environment? Do I need to have the standby in mount state? If dbca won't work does anyone have a manual procedure for adding a new instance?
Thankszulo
Let's say you adding node nusclust160## to you existing cluster and dbca is a pain to use.
Extend clusterware to the nusclust160## server.
re: Page 64 of Oracle® Clusterware Administration and Deployment Guide 11g Release 1 (11.1)
1a.
Add undo tablespace to support additional node.
Re-check space for DATA1 on nusclust16007 and /dbdata/ORADB on sun16109.
As of Thursday, May 21, 2009 the DATA1 asm group has 53,584M free.
As of Thursday, May 21, 2009 the /dbdata/ORADB has 77G free.
In a separate terminal window on nusclust16007 run the following in sqlplus
CREATE UNDO TABLESPACE UNDOTBS4 datafile '+DATA1' SIZE 13300M AUTOEXTEND ON ;
This will take a long time to create this tablespace. Please minimize the window after submitting the ddl and move on to the next step.
1b.
Insure .bash_profile on nusclust160## should look like this:
vi .bash_profile
export ORACLE_HOSTNAME=nusclust160##
export ORACLE_SID=ORADB4
export ORA_CRS_BASE=/apps/ocr/oracle
export ORACLE_BASE=/apps/dbs/oracle
export PATH=/usr/ccs/bin:/usr/X/bin:/usr/bin:/usr/sfw/bin:/usr/sbin:/usr/local/bin
export server=`uname -n`
export PS1="$ORACLE_SID@$HOSTNAME >"
alias cls='clear'
alias More='more'
alias ll='ls -lt | more'
Gather IP addresses for fourth node from /etc/hosts:
222.65.125.### nusclust160##
222.65.125.### nusclust160##-vip
10.333.248.### nusclust160##-priv
2. Start Oracle Universal Installer:
Go to CRS_home/oui/bin and run the addNode.sh script on one of the existing
nodes. Oracle Universal Installer runs in add node mode.
The Oracle inventory on nusclust16007, nusclust16008, and nusclust16036 are found under:
/home/oracle/oraInventory
Use a X windows enabled session (The OUI takes 33 minutes)
cd /apps/ocr/oracle/product/11.1.0/crs/oui/bin
./addNode.sh
a. In the first screen specify a new node as :
Public Node Name: nusclust160##
Private Node Name: nusclust160##-priv
Virtual Host Name: nusclust160##-vip
If you receive the error:
" tar. ./bin/racgvip.orig: Permission denied"
Do the following:
cd /apps/ocr/oracle/product/11.1.0/crs/bin
ls -al racgvip.orig
paste here:
chown root:oinstall racgvip.orig
chmod 771 racgvip.orig
should now show:
-rwxrwx--x 1 root oinstall 19213 Feb 11 08:36 racgvip.orig
As root:
a.
On nusclust160##:
cd /home/oracle/oraInventory
./orainstRoot.sh
b.
On nusclust16007:
cd /apps/ocr/oracle/product/11.1.0/crs/install
./rootaddnode.sh
clscfg: EXISTING configuration version 4 detected.
clscfg: version 4 is 11 Release 1.
Attempting to add 1 new nodes to the configuration
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 4: nusclust160## nusclust160##-priv nusclust160##
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
/apps/ocr/oracle/product/11.1.0/crs/bin/srvctl add nodeapps -n nusclust160## -A nusclust160##-vip/255.255.255.224/bge0
c.
On nusclust160##:
cd /apps/ocr/oracle/product/11.1.0/crs/
./root.sh
WARNING: directory '/apps/ocr/oracle/product/11.1.0' is not owned by root
WARNING: directory '/apps/ocr/oracle/product' is not owned by root
WARNING: directory '/apps/ocr/oracle' is not owned by root
Checking to see if Oracle CRS stack is already configured
OCR LOCATIONS = /raw/ocr/ocrconf1,/raw/ocr/ocrconf2
OCR backup directory '/apps/ocr/oracle/product/11.1.0/crs/cdata/rac_cluster' does not exist. Creating now
Setting the permissions on OCR backup directory
Setting up Network socket directories
Oracle Cluster Registry configuration upgraded successfully
The directory '/apps/ocr/oracle/product/11.1.0' is not owned by root. Changing owner to root
The directory '/apps/ocr/oracle/product' is not owned by root. Changing owner to root
The directory '/apps/ocr/oracle' is not owned by root. Changing owner to root
clscfg: EXISTING configuration version 4 detected.
clscfg: version 4 is 11 Release 1.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: nusclust16007 nusclust16007-priv nusclust16007
node 2: nusclust16008 nusclust16008-priv nusclust16008
node 3: nusclust16036 nusclust16036-priv nusclust16036
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Cluster Synchronization Services is active on these nodes.
nusclust16007
nusclust16008
nusclust16036
nusclust160##
Cluster Synchronization Services is active on all the nodes.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
4. After this is done crs_stat -t will show nusclust160## in the crs i.e.
I see:
Name Type Target State Host
ora....160##.gsd application ONLINE ONLINE sun...160##
ora....160##.ons application ONLINE OFFLINE
ora....160##.vip application ONLINE ONLINE sun...160##
Do not be concerned about ora.nusclust160##.ons being OFFLINE, as that will be fixed shortly in a step that follows this one.
5. As oracle :
On nusclust16007:
cd /apps/ocr/oracle/product/11.1.0/crs/bin
./racgons add_config nusclust160##:6251
This should take about one second to run.
If it says that it has already been added to the OCR you are fine.
If it hangs, you may need to reboot all servers to clear this issue.
6. Insure new node is properly added to ocr by running
On nusclust16007:
ocrdump
Check for the entries that show:
[DATABASE.ONS_HOSTS.nusclust160##.PORT]
ORATEXT : 6251
7. Check that your cluster is integrated and that the cluster is not divided into
partitions by completing the following operations:
On nusclust16007:
cd /apps/ocr/oracle/product/11.1.0/crs/bin
./cluvfy comp clumgr -n all -verbose
Should see Verification of cluster manager integrity was successful.
8.
Use the following command to perform an integrated validation of the Oracle
Clusterware setup on all of the configured nodes, both the preexisting nodes
and the nodes that you have added:
AS oracle on nusclust16007:
cluvfy stage -post crsinst -n all -verbose
Post-check for cluster services setup was successful.
good: Post-check for cluster services setup was successful.
9.
On nusclust160## as oracle run the following:
cd /apps/ocr/oracle/product/11.1.0/crs/bin
./crs_stat -t | grep OFFLINE
If you see this:
ora.nusclust160##.ons application ONLINE OFFLINE
then run this:
./crs_start -all
After:
./crs_stat -t
ora.nusclust160##.ons application ONLINE ONLINE nusclust160##
If you see the above then you can move on the next step.
Adding database binaries to the nusclust160## server and setting up the listener.
1.
From nusclust16007:
Open an X window (The OUI part takes 13 minutes)
cd /apps/dbs/oracle/product/11.1.0/db_1/oui/bin
./runInstaller -addNode ORACLE_HOME=/apps/dbs/oracle/product/11.1.0/db_1 $*
You should get a prompt to specify a new node, in this case you should see nusclust160## where you will need to put a check mark beside it.
2.
from nusclust160##:
Eventually you will be prompted to run the following as root on the new node
On nusclust160##
cd /apps/dbs/oracle/product/11.1.0/db_1
./root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /apps/dbs/oracle/product/11.1.0/db_1
Enter the full pathname of the local bin directory: [usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /var/opt/oracle/oratab file...
Entries will be added to the /var/opt/oracle/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
3. verification
Now set up the .bash_profile and .asm profile to on nusclust160## to support new ORADB4 and +ASM4 instances for the oracle userid.
On nusclust160##:
cp .bash_profile .bash_profile.bak
On nusclust16007:
sftp nusclust160##
put .bash_profile
On nusclust160##:
vi .bash_profile
change ORALCE_SID to ORADB4
cp .bash_profile .asm
vi .asm
change ORALCE_SID to +ASM4 in .asm file
which sqlplus
Should show the path below is $PATH environmental variable is set correctly.
/apps/dbs/oracle/product/11.1.0/db_1/bin/sqlplus
On nusclust160##:
oifcfg getif
This should show:
ce4 10.333.248.192 global cluster_interconnect
ce5 222.65.125.128 global public
4.
Run Netbackup Oracle Agent link script.
As oracle make sure ORACLE_HOME is fined.
env | grep ORACLE_HOME
then
cd /usr/openv/netbackup/bin/
./oracle_link
ls -al $ORACLE_HOME/lib/libobk.so
should show:
/apps/dbs/oracle/product/11.1.0/db_1/lib/libobk.so -> /usr/openv/netbackup/bin/libobk.so64.1
5.
On the target node, run the Net Configuration Assistant (NETCA) to add a
listener. Add a listener to the target node by running NETCA from the target node and
selecting only the target node on the Node Selection page.
I shall do the following on nusclust160## using X Windows
Now before I do this I see:
crs_stat -t
ora.nusclust160##.gsd application ONLINE ONLINE nusclust160##
ora.nusclust160##.ons application ONLINE ONLINE nusclust160##
ora.nusclust160##.vip application ONLINE ONLINE nusclust160##
Connect to nusclust160## and open up X windows session.
netca
Choose Cluster configuration.
select nusclust160## as the node to configure.
Choose Listener configuration, then Add.
When it prompts you for a listener name choose LISTENER as it will append _NUSCLUST160##(server name) to end of the LISTENER name to make a complete listener name.
At this point you will have listener to support the new node in the crs.
now
crs_stat -t
will show:
ora....0#.lsnr application ONLINE ONLINE nusclust160##
ora.nusclust160##.gsd application ONLINE ONLINE nusclust160##
ora.nusclust160##.ons application ONLINE ONLINE nusclust160##
ora.nusclust160##.vip application ONLINE ONLINE nusclust160##
At this point the necessary crs entries for gsd, ons, vip, and the listener on nusclust160## all we need now are the ORADB4 and +ASM4 instances added.
III. 7/11/2009 7:40 AM Sat [120 min] NTTA DBA
Use NON dbca method to create additional instances on the nusclust160## server. This will involve a complete shutdown of all RAC instances.
1.
Undo tablespace creation was taken care of in Step I,1. Check on the progress of the creation of tablespace UNDOTBS4 in the minimized window. Should see tablespace on primary and physical standby databases.
2. First we shall set up the +ASM4 instance on nusclust160## and add it to the cluster.
On nusclust160##
cd $ORACLE_HOME/dbs
vi init+ASM4.ora
# Copyright (c) 1991, 2001, 2002 by Oracle Corporation
# Cluster Database
cluster_database=true
cluster_database_instances=6
# Miscellaneous
diagnostic_dest=/apps/dbs/oracle
instance_type=asm
# Pools
large_pool_size=12M
asm_diskgroups='DATA1','ARCH','REDO1','REDO2'
asm_diskstring='/raw/asm'
+ASM1.instance_number=1
+ASM2.instance_number=2
+ASM3.instance_number=3
+ASM4.instance_number=4
3.
On nusclust16007
cd $ORACLE_HOME/dbs
sftp nusclust160##
put orapw+ASM1 /apps/dbs/oracle/product/11.1.0/db_1/dbs
put orapwORADB1 /apps/dbs/oracle/product/11.1.0/db_1/dbs
4.
On nusclust160##
cd $ORACLE_HOME/dbs
cp orapw+ASM1 orapw+ASM4
cp orapwORADB1 orapwORADB4
5.
On nusclust160##
cd $HOME
. ./.asm
sqlplus '/ as sysasm'
startup
create spfile from pfile='/apps/dbs/oracle/product/11.1.0/db_1/dbs/init+ASM4.ora' ;
shutdown immediate ;
startup
show parameters spfile
6. Now that we have a running asm instance add it the cluster.
On nusclust160##
srvctl add asm -n nusclust160## -i +ASM4 -o /apps/dbs/oracle/product/11.1.0/db_1
srvctl enable asm -n nusclust160## -i +ASM4
7. Now that we have an asm instance let's set up a database instance.
On nusclust16007/ORADB1 :
alter system set cluster_database_instances=6 scope=spfile ;
alter system set instance_name=ORADB4 scope=spfile sid='ORADB4' ;
alter system set instance_number=4 scope=spfile sid='ORADB4' ;
alter system set local_listener=LISTENER_ NUSCLUST160## scope=both sid='ORADB4' ;
alter system set thread=4 scope=both sid='ORADB4' ;
alter system set undo_tablespace=UNDOTBS4 scope=both sid='ORADB4' ;
alter database add logfile thread 4 group 28 ('+REDO1', '+REDO2' ) size 100M ;
alter database add logfile thread 4 group 29 ('+REDO1', '+REDO2' ) size 100M ;
alter database add logfile thread 4 group 30 ('+REDO1', '+REDO2' ) size 100M ;
alter database add logfile thread 4 group 31 ('+REDO1', '+REDO2' ) size 100M ;
alter database enable public thread 4;
Need to add 5 groups to support standby
So at the end of the day 900M will be added to REDO1(29,577M free) and 900M will be added to REDO2 (29,577M free).
8. Set up init.ora, listener.ora, and tnsnames.ora for ORADB4 on nusclust160##.
a. init.ora set up
cd $ORACLE_HOME/dbs
vi initORADB4.ora
SPFILE='+DATA1/ORADB/spfileORADB.ora'
b. add entries to tnsnames.ora:
ORADB4 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = ORADB)
(INSTANCE_NAME = ORADB4)
ORADB =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16007-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16008-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16036-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
(LOAD_BALANCE = yes)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = ORADB)
LISTENERS_ORADB =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16007-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16008-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16036-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
LISTENER_NUSCLUST160## =
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
ORADB_PRIM =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16007-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16008-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16036-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
(LOAD_BALANCE = yes)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = ORADB)
c. add entries to listener.ora, The entries for most of this file should be set already, just insure modifications that need to be made are made.
SID_LIST_LISTENER_NUSCLUST160## =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = /apps/dbs/oracle/product/11.1.0/db_1)
(PROGRAM = extproc)
(SID_DESC =
(GLOBAL_DBNAME = ORADB)
(ORACLE_HOME = /apps/dbs/oracle/product/11.1.0/db_1)
(SID_NAME = ORADB4)
LISTENER_NUSCLUST160## =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = NUSCLUST160##-vip)(PORT = 1521)(IP = FIRST))
(ADDRESS = (PROTOCOL = TCP)(HOST = 222.65.125.###)(PORT = 1521)(IP = FIRST))
9. Reload the listener.
lsnrclt
set current_listener LISTENER_NUSCLUST160##
reload
exit
10. Check audit trail, add instance to cluster, and start db instance.
a.
Check for audit directory and start the instance.
/apps/dbs/oracle/product/11.1.0/db_1/rdbms/audit
If this audit trail directory does not exist then create it.
b.
srvctl add instance -d ORADB -i ORADB4 -n nusclust160##
srvctl modify instance -d ORADB -i ORADB4 -s +ASM4
srvctl enable instance -d ORADB -i ORADB4
Will probably show: PRKP-1017 : Instance ORADB4 already enabled.
c.
sqlplus '/ as sysdba'
startup
**Because the cluster_database_instances parameter requires the complete shutdown of all instances in the cluster, you might have an issue when it attempts to start the instance. If you receive an error then run:
srvctl stop database -d oradb
sqlplus '/ as sysdba'
startup
shutdown
srvctl start database -d oradb
shutdown
srvctl start instance -d ORADB -i ORADB4 -o open
11.
Modify spfile of ASM1, ASM2, +ASM3
On nusclust16007
. ./.asm
sqlplus '/ as sysasm'
alter system set instance_number=4 scope=spfile sid='+ASM4' ;
On nusclust16008
. ./.asm
sqlplus '/ as sysasm'
alter system set instance_number=4 scope=spfile sid='+ASM4' ;
On nusclust16036
. ./.asm
sqlplus '/ as sysasm'
alter system set instance_number=4 scope=spfile sid='+ASM4' ;
b Modify tnsnames.ora on nusclust nusclust16007, nusclust16008, and nusclust16036
On nusclust16007
ORADB4 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = ORADB)
(INSTANCE_NAME = ORADB4)
Add the following line to the ORADB alias:
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
Add the following line to the LISTENERS_ORADB alias:
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
Add the following line to the ORADB_PRIM alias:
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
On nusclust16008
ORADB4 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = ORADB)
(INSTANCE_NAME = ORADB4)
Add the following line to the ORADB alias:
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
Add the following line to the LISTENERS_ORADB alias:
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
Add the following line to the ORADB_PRIM alias:
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
On nusclust16036
ORADB4 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = ORADB)
(INSTANCE_NAME = ORADB4)
Add the following line to the ORADB alias:
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
Add the following line to the LISTENERS_ORADB alias:
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
Add the following line to the ORADB_PRIM alias:
(ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
c Add standby logs on primary to support 4th node.
alter database add standby logfile thread 4 group 32 ('+REDO1', '+REDO2' ) size 100M ;
alter database add standby logfile thread 4 group 33 ('+REDO1', '+REDO2' ) size 100M ;
alter database add standby logfile thread 4 group 34 ('+REDO1', '+REDO2' ) size 100M ;
alter database add standby logfile thread 4 group 35 ('+REDO1', '+REDO2' ) size 100M ;
alter database add standby logfile thread 4 group 36 ('+REDO1', '+REDO2' ) size 100M ;
12.
Test the cluster to make sure everything is set up correctly.
a. Shutdown resources.
On nusclust16007:
emctl stop dbconsole
ps -ef | grep perl
ps -ef | grep agent
ps -ef | grep java
On nusclust16008:
emctl stop dbconsole
On nusclust16036:
emctl stop dbconsole
On nusclust16008:
cd $HOME
. ./.rman
cd scripts
./go
shutdown immediate
cd $HOME
. ./.bash_profile
srvctl stop database -d oradb
crs_stop -all
crs_stat -t
b. Startup resources
On nusclust16007:
cd $HOME
. ./.bash_profile
crs_start -all
crs_stat -t
The command above should show everything up and running.
ocrcheck
On nusclust16008:
cd $HOME
. ./.rman
cd scripts
./go
startup
On nusclust16007:
emctl start dbconsole
On nusclust16008:
emctl start dbconsole
On nusclust16036:
emctl start dbconsole
How does that work for you?
-JR jr -
Hi.
We have a RAC environment with two nodes running on Linux 64 bit(10.1.0.3).
This RAC invironment is installed with Db controle and have both intelligent agent and dbconsole running on both nodes.
We have now installed Grid control on another linux box with 10.1.0.3 Repository database, and we want to discover these RAC nodes from the Management Server.
Do we need to install a new Intelligent Agent into a separate ORACLE_HOME or can the agent installed during the default RAC installation be used. Further, if we need to install a new Agent, do we have to stop both agent and dbconsole running in the RAC Home and do we have to clean up the sysman schema in the RAC database?
Any help is apreciated.
Rgds
Kjell OveMake sure emd.properties has
# OMS Upload URL
# if there is no receiving OMS or if you wish to disable the UploadManager
# please set this value to empty or comment out below line
REPOSITORY_URL=http://xxxxxxx:4889/em/upload/
where xxxxx points to your Grid Control.
Also, set the following:
# emagent perl tracing levels
# supported levels: DEBUG, INFO, WARN, ERROR
# default level is ERROR
EMAGENT_PERL_TRACE_LEVEL=DEBUG
# other optional parameters for perl tracing
# EMAGENT_PERL_TRACE_DIR default to $ORACLE_HOME/sysman/log
EMAGENT_PERL_TRACE_DIR=/tmp
This will create a .trc file in /tmp to show you what is happening.
Verify that $ORACLE_HOME/bin/emctl has the following correctly set to your CRS directory
CRS_HOME=/opt/app/oracle/product/10.1.0/crs
$CRS_HOME/bin/cemutlo -n
must return the name of your cluster.
From OEM Grid, click on the agent for the RAC host, then click "Add Database" Go. This will force a re-discover and then it should also populate the above trace file. Post the above trace file if it still is not working.
Maybe you are looking for
-
How do I set an exclusive password for the iCloud settings in my Mac at work?
Here's the thing: I want to have my computer at work (MacPro) synched with iCloud (all my contacts and bookmarks). But to do and still feel safe that no one else would open System Preferences and mess up with the iCloud settings (for example, turning
-
Periodically I push the start button and nothing happens...Try on and off for a couple of hours and then, it boots up!????
-
Don't know what details you might want.
-
Please help! I cant seem to get rid of the default theme
I have a completed iphoto slideshow with music in the scrapbook theme. I then send it to idvd to burn anad it keeps coming up with the default theme as well........I need it for a funeral.......any ideas where I am going wrong? cheers Jan
-
Slow start up and almost frozen finder
please can anyone help me on how to reset my computer or something???? My problem is it takes about 1 hour from when i turn my Powerbook G4 on till it gets into finer and even when i get to finder it takes about 2 min to open my HD folder and my appl