OSB cluster networking

Question

We're attempting to implement OSB in a highly-available configuration. We're running WLS 10.3.4/OSB 11.1.1.4 on 64bit OEL 5.4/SunJava 1.6.0_23.
We have OSB running standalone in a number of environments, but need a fault-tolerant design.
Per the ["Whole Server Migration"|http://download.oracle.com/docs/cd/E11035_01/wls100/cluster/migration.html] document, we've setup SSH-based node manager as required. We use this same configuration on our standalone systems without issue - works great.
We've installed two systems and set up a clustered domain using shared storage on NFS. The share stores a single middleware home as well as the domain directory. If we configure the VIPs manually on the machines, the admin managed servers will start, but we're obviously missing some key info here, as there is no mention we can find in any documentation that explains how to set up the NIC/VIP failover. [This section|http://download.oracle.com/docs/cd/E17904_01/core.1111/e10106/ha_soa.htm#CHDJHHFI] appears to address it, but it appears to be referring to the java-based node manager, not the script-based node manager. (This is a common thread we've noticed in the documentation on this subject - casual inter-mixing of referrences to both solutions without distinction when they appear to have nothing to do with one-another.)
Attempting to follow the documentation results in circular references, confusion, paralysis, and TONS of wasted time and frustration.
So...What are we missing?
Edit: Oh wait - there's more - so [This iteration|http://download.oracle.com/docs/cd/E14571_01/core.1111/e12036/server_migration.htm#insertedID0] appears to follow many similar steps, yet once again seems to refer to the java-based node manager, (nodemanager.properties,) then proceeds to refer to the script-based node manager (wlscontrol.sh) just a few lines down. Seriously, guys.  What the hell is going on with this stuff!?
Edited by: Scott555 on Jul 8, 2011 8:11 AM

Answer 1

Some background information: The SSH node manager started out as a simplified version of the node manager to address the challenges
of the Java node manager's requirement for using two-way SSL to communicate with the admin server. In WebLogic Server 9.x and 10.0,
the SSH version was the only one that supported some more advanced features (for example, Automatic Whole Server Migration); however,
this changed in WebLogic Server 10.3. The SSH version is only supported on Unix-based operating systems. Given that the Java node manager
no longer uses two-way SSL, is feature comparable, and provides additional security.
Some information on the SSH node manager can be found here: http://download.oracle.com/docs/cd/E12839_01/web.1111/e13740/script_nodemgr.htm
the requirements for using whole server migration (WSM) are:
- WSM uses a floating IP address, also known as a virtual IP address, for each migratable server. This means that the migratable server candidate machines have to be in the same subnet
(because the virtual IP address must be valid on all candidate machines).
- WSM requires the use of the node manager. You must make sure the node manager on each candidate machine is properly initialized with the security-related files it need to authenticate and
accept commands from the admin server.
- WSM uses the node manager to migrate the virtual IP address and assign it to the target machine. As such, the default configuration assumes that the machines are similar; specifically,
it assumes the following:
- The netmask associated with the virtual IP is the same on all machines.
- The network device name (for example, eth0 on Linux) is the same on all machines.
- The functional behavior of the platform-specific OS command used to add and remove the virtual IP (for example, ifconfig on Linux) is the same.
- WSM only supports migration of a single virtual IP address for each migratable server. Therefore, a migratable server cannot define any network channels that use a Listen Address different
from the virtual IP address associated with the server. If you need your servers to use multiple network channels associated with multiple IP addresses, you cannot use theWSM framework.
- WSM assumes that any server-specific state is already shared through some highly available sharing mechanism. For example, the server’s default persistent store where it keeps its XA
transaction logs must be accessible on all candidate machines using the exact same path.
The steps required to set up automatic whole server migration are
- Create your domain. Make sure that you set up each managed server’s to Listen Address to its virtual IP address and assign it to a machine.\
- Set up the node manager for each candidate machine. For each machine, edit the nodemanager.properties file to set the NetMask property to the netmask associated with
the virtual IP addresses being used and Interface to the network device name with which to associate the virtual IP address. Typically, the nodemanager.properties file is created
the first time the node manager is started in the $NODEMGR_HOME directory (by default, $WL_HOME/common/nodemanager).
- Verify the domain and node manager configuration. Before we proceed, start up the domain and each clustered managed server via its node manager. This not only ensures that the
node managers and servers are properly configured but also initializes the node managers with the password files they need to accept commands from the admin server. Don’t forget
to start managed servers on all candidate machines to ensure that the node manager and domain directory are properly initialized.
- Choose and configure your leasing mechanism. Like automatic service migration, automatic whole server migration relies on leasing. Use the cluster’s Migration Configuration tab to
select the appropriate Migration Basis.
- If you choose database leasing, be sure to create and configure your nontransactional data source, create the leasing table, and use the cluster’s Migration Configuration tab to set the
Data Source for Automatic Migration and Auto Migration Table Name appropriately.
- Grant superuser privileges to the wlsifconfig script. Node managers use the $WL_HOME/common/bin/wlsifconfig.sh (onWindows, wlsifconfig.cmd) script to add and
remove virtual IP addresses from the machines. By default, the file is set up to use sudo; sudo typically prompts you for your password the first time you run it and periodically after that.
To do this without needing to input your password, you need to add the NOPASSWD option to the relevant entry in your /etc/sudoers file. Don’t forget to add the wlsifconfig script to
your PATH so that the node managers can locate it. weblogic machine1 = NOPASSWD: /oracle/middleware/wlserver_10.3/common/bin/wlsifconfig.sh
- Enable automatic server migration. The last step is to use each managed server’s Migration Configuration tab to select the Automatic Server Migration Enabled checkbox and restart the servers.
At this point, automatic whole server migration configuration is complete and needs to be tested. Debugging problems with whole server migration can be tricky so you will probably want to add
-Dweblogic.debug.DebugServerMigration=true to the Java command line used to start your servers.
The document for WebLogic 10.3.4 whole server migration can be found here: http://download.oracle.com/docs/cd/E17904_01/web.1111/e13709/migration.htm#BABFEFEG
Note the remark in the documentation: The Java version of Node Manager can be used for server migration on Windows or UNIX. The SSH version of Node Manager can be used for server migration on UNIX only.