Cluster Setup hung

Hi All,
          we are running in production environment. This case is very very critical. I checked thrice the cluster configuration everything seems to be fine. We are using WebLogic Server 5.1 with SP11 in a clustered environment. The operating system is Sun Solaris 2.6. The database version is Oracle 8.1.7 (Oracle parallel server).
          Hardware Cluster : Sun Cluster 3.0
          Solaris_JDK 1.2.2_05a
          IPlanet Web Server 4.1 - proxy plug-in
          very often both the server goes for a hung state and we are not able to send any request and recieve any response from both HTTP and T3. Is I am missing out something? Or Is this a bug with BEA WebLogic?
          please throw me some light. Any help is really appreciated.
          Many Thanks in Advance,
          C R Baradwaj

Raghuram Bharadwaj wrote:
Hi All,
I checkted thrice the cluster configuration everything seems to be fine.
We are using WebLogic Server 5.1 with SP11 in a clustered environment. The operating
system is Sun Solaris 2.6. The database version is Oracle 8.1.7 (Oracle parallel
server).
Hardware Cluster : Sun Cluster 3.0
Solaris_JDK 1.2.2_05a
IPlanet Web Server 4.1 - proxy plug-in
very often both the server goes for a hung state and we are not able to send any
request and recieve any response from both HTTP and T3.Hi. This sounds like you should get official support for, with a case number etc.
At present, there is nothing in your post to indicate that JDBC-related problems
are the key (yet).
Joe
>
>
we even tried testing the multicast address using java utils.MulticastTest, when
the server hungs. This works fine.
The weblogic.log, proxy.log & access.log didin't show any error messages. Even
the thread dump(kill -3) doesn't carry any messages. JDBC Connection Pool is working
fine. we checked thru T3AdminJDBC. All connections are released.
We tried all the possibilities but we couldn't see any errors anywhere. But we
are unable to do even ping to the weblogic server.
java weblogic.Admin t3://<ip_address>:port_no PING 1 1
even this fail.
Is I am missing out something? Or Is this a bug with BEA WebLogic?
please throw me some light. Any help is really appreciated.
Many Thanks in Advance,
C R Baradwaj

Similar Messages

OSB cluster setup with SSL

Hi,
Could any one help me here..
Cluster setup for OSB with SSL enable
1) Admin and 2 Managed server are running on same host
2) cluster domain created in development mode
2) While starting second managed server getting below error..
<Oct 4, 2010 8:04:58 AM PDT> <Error> <ClusterTimer> <BEA-000000> <Cannot contact Admin server. Therefore constructing the Cluster Authority Current time with
the time skew 0
java.rmi.RemoteException: ClusterTimerAuthority error; nested exception is:
javax.naming.CommunicationException [Root exception is java.net.ConnectException: t3s://hostname:SSLport: Destination unreachable; nest
ed exception is:
java.io.IOException: Stream closed.; No available router to destination]
at com.bea.wli.sb.init.RemoteClusterTimerAuthority.getClusterTimerAuthorityCurrentTime(RemoteClusterTimerAuthority.java:38)
at com.bea.wli.timer.ClusterTimerService.clusterTimeAuthorityCurrentTimeMillis(ClusterTimerService.java:177)
at com.bea.wli.timer.ClusterTimerService.initialize(ClusterTimerService.java:88)
at com.bea.wli.sb.init.FrameworkStarter._preStart(FrameworkStarter.java:221)
at com.bea.wli.sb.init.FrameworkStarter.access$000(FrameworkStarter.java:79)
Truncated. see log file for complete stacktrace
Thanks,
Sushma.

Even I faced the same issue..but eventually the problem got resolved with below resolution:
Resolution: The managed server was not able to connect to t3s://hostname:sslport. The SSL configuration on Adminserver was wrong. After correcting SSL setting on Adminserver..i was able to resolve this error.

Verify HA Cluster setup in Oracle Virtual machine thru GUI

Hi All,
I have setup 3 node cluster with HA[Oracle VM] enabled.When i checked in the GUI it
shows "High availability Infrastructure works well" like that.
For testing HA, I poweroff one of the Oracle Virtual server node from the Oracle Virtual manager browser.
Then if i checked the "Server Pools > Edit Server Pool ".check for "High Availability Infrastructure" browser keep going.
Also one more thing is the Powered off machine automatically after some time it is coming up.Is
it right ?
After powered off if i run the below command in ovmmgr it shows 2 nodes are up.Same o/p
how can i get in GUI when i clicked High Availability Infrastructure in "Server Pools > Edit Server Pool "
[root@xxxovmsr1 ~]# /usr/lib/ovs/ovs-cluster-check master alter-fstab
O2CB cluster ocfs2 already online
2
Cluster setup complete.
Link hangs quit sometime.Not allowing new connections also ..What to do ?
Basic question
How to verify the cluster setup from the GUI when i click "High Availability Infrastructure" check button?
Some one can help me in this please ?
Regards,
JK
Edited by: user2861234 on Jun 29, 2009 8:11 AM

Hey,
After powered off one of VM server if i run the below command in ovmmgr it shows 2 nodes are up.
Why are you going ovmmgr ?
Are your checking it only thru gui part of the manager or also thru thr command line.
Oracle vm server - are hosts for your virtual systems your create on them.
If you power off the server - from GUI part you can see - it as switched off or unreachable
If your HA feature is enabled for that pool having more than 1 server sharing the /OVS filesystem - then he virtual system which is runnign on 1 of the Oracle VM server - will not die
It will migrate to other available server in the pool - which is whole idea and purpose of HA
I don't understand what exactly you are trying to crosscheck or verify here with these commands
High Availability Infrastructure in "Server Pools > Edit Server Pool "
root@xxxoxxxr1 ~# /usr/lib/ovs/ovs-cluster-check master alter-fstab
O2CB cluster ocfs2 already online
2
Cluster setup complete.
If you have to check whether your HA feature is working - form GUI you can got o server pool - tab -
edit it
and chk the HA - tab
and verify-
it HA feature is working properly - it will give you message accordingly.

Why do we use reverse proxy for Oracle RAC Cluster setup

Hello All,
I got this question lately.. "why do we use reverse proxy for Oracle RAC Cluster setup". I know we use the reverse proxy at Middleware level for multiple security reasons.
Thanks..

"why do we use reverse proxy for Oracle RAC Cluster setup".
I wouldn't. I wouldn't use a proxy of any sort for the Cluster Interconnect for sure.
Cheers,
Brian

BAM Cluster setup - issue with whole server migration configuration

HI,
We are working on a SOA , BAM , BPM Cluster setup on OEL VMs
With Admin server running on Host1 and Soa_server1, BAM_server1 , Host2 - Soa_server2 , Bam_server2.
We were successfully able to setup clustering for Soa without using the webTier utilities in our environment, Just by adding entries in the /etc/hosts file.
Followed the document - except for web tier setup
http://www.namredlek.nl/orasoa/InstallSOASuite11gOnCluster_v1.pdf
1)     Installed Weblogic 11g.
2)     Soa 11g.
3)     Domain creation
4) Configured Node manager
5) Soa Clustering completed successfully.
6) Domain up and running with SOA_Cluster up and running.
7)     Extended the domain with BAM
The issue revolves when we are doing teh Migration configurations from the console for BAM and we restart BAM server we get an error -
Apr 14, 2011 2:30:35 AM weblogic.nodemanager.server.Handler handleStart
WARNING: Exception while starting server 'bam_server1'
java.io.IOException: Command '/oracle/Shared_MWHome/user_projects/domains/SOA_Domain/bin/server_migration/wlsifconfig.sh -addif eth0 192.168.1.6 255.255.255.0 ' returned an unsuccessful exit code '1'. Check NM logs for script output.
and Bam server goes to Failed Not Restartable state.

The error is clearly out there
java.io.IOException: Command '/oracle/Shared_MWHome/user_projects/domains/SOA_Domain/bin/server_migration/wlsifconfig.sh -addif eth0 192.168.1.6 255.255.255.0 ' returned an unsuccessful exit code '1'. Check NM logs for script output.
This error occurs due to a failure in adding the virtual IP of BAM server to the network interface you have specified in nodemanager.properties.
You need to check whether you have specified the correct interface and subnet mask in node manager.properties as below :
The fourth step is to edit the Node Manager's properties file. This file is called nodemanager.properties
and is located in the ORACLE_BASE/product/osbmw/wlserver_10.3/common/nodemanager directory.
For server migration to work properly, you need to add the properties listed below:
Interface=eth0
NetMask=255.255.255.0
UseMACBroadcast=true
· Interface
This property specifies the interface name for the floating IP (for example, eth0).
· NetMask
This property specifies the net mask for the interface for the floating IP. The net mask should the
same as the net mask on the interface; 255.255.255.0 is used as an example in this document.
· UseMACBroadcast
This property specifies whether or not to use a node's MAC address when sending ARP packets,
that is, whether or not to use the -b flag in the arping command.

SOAP ports with cluster setup and BES?

Okay, our cluster setup in NetWare (yes, still NetWare cluster) with Groupwise 7.0.3 we have unique TCP ports for the client/server (like 1677-something). Even though each clustered node has a unique IP and is running in protected memory.
That being said:
We'd like to enable SOAP for use with BES and GWArchive.
The issue/question:
Do either of these figure out the SOAP port for each PO? Or must the SOAP port be the same among all the PO's for BES to work properly?
If the SOAP port must be the same (I'm not the BES admin, so I don't know if the setup lets you specify just one port or if you can defined them per PO), can you run multiple POA on the same physical host (keep in mind, this is clustered with protected memory) and use the same TCP port # for SOAP?

kjhurni wrote:
> Okay, our cluster setup in NetWare (yes, still NetWare cluster) with
> Groupwise 7.0.3 we have unique TCP ports for the client/server (like
> 1677-something). Even though each clustered node has a unique IP and is
> running in protected memory.
>
> That being said:
>
> We'd like to enable SOAP for use with BES and GWArchive.
>
> The issue/question:
>
> Do either of these figure out the SOAP port for each PO? Or must the
> SOAP port be the same among all the PO's for BES to work properly?
>
> If the SOAP port must be the same (I'm not the BES admin, so I don't
> know if the setup lets you specify just one port or if you can defined
> them per PO), can you run multiple POA on the same physical host (keep
> in mind, this is clustered with protected memory) and use the same TCP
> port # for SOAP?
>
>
I haven't used BES, but in general there's no auto-discover method for
SOAP ports, so yes they'd probably have to be the same port.

How to Uninstall SQL instance on active-passive SQL server , which failed during Cluster Setup (Error-Failed at Validate Active Directory Configuration)

How to Uninstall SQL instance on active-passive SQL server , which failed during Cluster Setup (Error-Failed at Validate Active Directory Configuration)
active-passive SQL server cluster setup failed due to some steps missed in initial cluster setup,
now i have unistall sql instance from nodes,
Your help will higly appriciated.
Regards,
Anish
Asandeen

Hello,
Please refer to the following link about remove a node of SQL Server Failover Cluster Instance:
http://msdn.microsoft.com/en-us/library/ms191545.aspx#Remove
Regards,
Fanny Liu
Fanny Liu
TechNet Community Support

Display logfiles in cluster setup (HTTPS UI)

Hi all
In my standalone setup, I can display and download logfiles in HTTPS-Webgui under "Log Subscriptions" -> "Log Files".
But in a cluster setup, I can't see the column "Log Files" under "Log Subscriptions". How can I access the files?
Thanks!
Christoph

Hello Christoph,
viewing individual logfiles on the GUI is notavailable for appliances in a cluster. You can still access them via your browser by using:
ftp://appliancename.local
as URL, where "appliancename.local" is the hostname or IP. This will prompt you for an username and password, you can use your normal admin credentials. Also possible:
ftp://[email protected]
ftp://admin:[email protected]
If you want to include username (and password)
Hope that helps,
Andreas

Network Extender & Cluster Setup

Does the Ingress unit satisfy the GPS signal requirements for other non-Ingress units in a cluster setup?
We have a large steel warehouse that has no cell coverage in the building. I can provide coverage with 3 Network Extenders. However, my problem is going to be the GPS antenna's. I have 1 window that provides a good GPS signal for the 1 extender I have now. If I purchase 2 additional extenders to complete the AoC will each of them have to lock on to GPS too? Or do the non-ingress units not require GPS. It may be wishful thinking, but our only other option is to cut a hole to access the outside & I really dont think these antenna' can handle the environment.
All units will be on the same network switch.

Ah. And I am guessing (really just confirming) that Open Directory is one piece of functionality which is in OSX Server, but not in the basic Snow Leopard distribution.
Sooo, is it worth digging around to see if any of the linux distributions out there contain Open Directory? I have an old lime (333 MHz) iMac in the basement...

Weblogic cluster setup

How can I setup two machines in cluster?
What all I need to setup and whats the procedure.

http://e-docs.bea.com/wls/docs70/pdf/cluster.pdf
http://e-docs.bea.com/wls/docs70/cluster/setup.html

Oracle cluster setup

Hi There,
We're trying to build a clustered oracle environment using oracle 11g (11.2.0.1) under Windows 2008 server.
The basic setup is to have 2 servers (active/passive), one shared SAN disk, install the software on both servers, put the database datafiles on the shared disk/SAN and when needed we can switch the shared disk between the 2 servers as failover/failback.
IS the above industry standard cluster setup? or is there a better way of doing it? does oracle/windows provide a way to failover/failback between servers?
Are there any documents that we can read that explains a good cluster setup please (without using RAC)? Or is there a document that have a better setup?
Can someone shed some light on this please and/or point us to the right direction.
Thanks

Does oracle clusterware comes shipped with the specific release (10g, 11g..etc) database installation pack or is it a separate download?
We saw this under the oracle download section:
Oracle Database 11g Release 2 Grid Infrastructure (11.2.0.1.0) for Microsoft Windows (x64)
Download win64_11gR2_grid.zip (715,166,425 bytes) (cksum - 3127109177)
Contains the Grid Infrastructure Software including Oracle Clusterware, Automated Storage Management (ASM), and ASM Cluster File System. Download and install prior to installing Oracle Real Application Clusters, Oracle Real Application Clusters One Node, or other Oracle software in a Grid Environment So, I was wondering if we should download this and install it on the server?
Thanks
Edited by: rsar001 on Jul 19, 2010 8:20 AM

Kernel Upgrade of R/3 4.7 Enterprise in Cluster Setup

Hi Experts,
We have a cluster setup with SAP R/3 Enterpirse 4.7 with MSSQL in windows 2003 server with node A and node B. Presently the system is running on Node B.
We also have two application servers (Dialog Instance).
Now I want to upgrade kernel of R/3 system. Please let me know what is the procedure for upgrading the kernel in cluster setup.
Appreciate quick response.
Regards
Sharath

Hi All,
I have successfully upgraded the kernel patch from 640-43 to 640-201 for R/3 4.7 with cluster setup. Currently the system was in Node B.
The only problem which I faced after copying the files to run folder was that the system was not coming up. So we copied back the old kernel files and started the system and it was coming online.
Wen we anayzed the problem we came to know that after copying the latest kernel files the system was not coming up as some DLLs were missing. The Note 684106 - Microsoft runtime DLLs solved the issue. Downloaded the files R3DLLINSTZIP which was given in the note and installed the files and after that my system came up with the latest kernel files.
Hope this will be helpfull for someone in futue.
Regards
Sharath Babu

Database bau host in HA cluster setups

In cluster setups for MSSQL, is there a technical term to describe the host that will run specific SQL instances/databases under normal circumstances (i.e. while it is up and doesnt need to be run by another server in the cluster)?

This is the "Primary Node" for the instance. That's not a term that has any technical meaning in a Windows Cluster. You may use cluster failover policies for the instance to help ensure that the instance is normally running on its Primary Node.
You can set the Primary Node for the instance as the "Preferred Owner" in the cluster configuration. This will cause that node to be preferred over other nodes for failover
and failback.
David
David http://blogs.msdn.com/b/dbrowne/

Simple two node Cluster Install - Hung after reboot of first node

Hello,
Over the past couple of days I have tried to install a simple two node cluster using two identical SunFire X4200s, firstly following the recipe in: http://www.sun.com/software/solaris/howtoguides/twonodecluster.jsp
and when that failed referring to http://docs.sun.com/app/docs/doc/819-0912 and http://docs.sun.com/app/docs/doc/819-2970.
I am trying to keep the install process as simple as possible, no switch, just back to back connections for the internal networking (node1 e1000g0 <--> node2 e1000g0, node1 e1000g1 <--> node2 e1000g1)
I ran the installer on both X4200s with default answers. This went through smoothly without problems.
I ran scinstall on node1, first time through, choosing "typical" as suggested in the how to guide. Everything goes OK (no errors) node2 reboots, but node1 just sits there waiting for node2, no errors, nothing....
I also tried rerunning scinstall choosing "Custom", and then selecting the no switch option. Same thing happened.
I must be doing something stupid, it's such a simple setup! Any ideas??
Here's the final screen from node1 (dcmds0) in both cases:
Cluster Creation
Log file - /var/cluster/logs/install/scinstall.log.940
Checking installation status ... done
The Sun Cluster software is installed on "dcmds0".
The Sun Cluster software is installed on "dcmds1".
Started sccheck on "dcmds0".
Started sccheck on "dcmds1".
sccheck completed with no errors or warnings for "dcmds0".
sccheck completed with no errors or warnings for "dcmds1".
Configuring "dcmds1" ... done
Rebooting "dcmds1" ...
Output from scconf on node2 (dcmds1):
bash-3.00# scconf -p
Cluster name: dcmdscluster
Cluster ID: 0x47538959
Cluster install mode: enabled
Cluster private net: 172.16.0.0
Cluster private netmask: 255.255.248.0
Cluster maximum nodes: 64
Cluster maximum private networks: 10
Cluster new node authentication: unix
Cluster authorized-node list: dcmds0 dcmds1
Cluster transport heart beat timeout: 10000
Cluster transport heart beat quantum: 1000
Round Robin Load Balancing UDP session timeout: 480
Cluster nodes: dcmds1
Cluster node name: dcmds1
Node ID: 1
Node enabled: yes
Node private hostname: clusternode1-priv
Node quorum vote count: 1
Node reservation key: 0x4753895900000001
Node zones: <NULL>
CPU shares for global zone: 1
Minimum CPU requested for global zone: 1
Node transport adapters: e1000g0 e1000g1
Node transport adapter: e1000g0
Adapter enabled: no
Adapter transport type: dlpi
Adapter property: device_name=e1000g
Adapter property: device_instance=0
Adapter property: lazy_free=1
Adapter property: dlpi_heartbeat_timeout=10000
Adapter property: dlpi_heartbeat_quantum=1000
Adapter property: nw_bandwidth=80
Adapter property: bandwidth=70
Adapter port names: <NULL>
Node transport adapter: e1000g1
Adapter enabled: no
Adapter transport type: dlpi
Adapter property: device_name=e1000g
Adapter property: device_instance=1
Adapter property: lazy_free=1
Adapter property: dlpi_heartbeat_timeout=10000
Adapter property: dlpi_heartbeat_quantum=1000
Adapter property: nw_bandwidth=80
Adapter property: bandwidth=70
Adapter port names: <NULL>
Cluster transport switches: <NULL>
Cluster transport cables
Endpoint Endpoint State
Quorum devices: <NULL>
Rob.

I have found out why the install hung - this needs to be added into the install guide(s) at once!! - It's VERY frustrating when an install guide is incomplete!
The solution is posted in the HA-Cluster OpenSolaris forums at:
http://opensolaris.org/os/community/ha-clusters/ohac/Documentation/SCXdocs/relnotes/#bugs
In particular, my problem was that I selected to make my Solaris install secure (A good idea, I thought!). Unfortunately, this stops Sun Cluster from working. To fix the problem you need to perform the following steps on each secured node:
Problem Summary: During Solaris installation, the setting of a restricted network profile disables external access to network services that Sun Cluster functionality uses, ie: The RPC communication service, which is required for cluster communication
Workaround: Restore external access to RPC communication.
Perform the following commands to restore external access to RPC communication.
# svccfg
svc:> select network/rpc/bind
svc:/network/rpc/bind> setprop config/local_only=false
svc:/network/rpc/bind> quit
# svcadm refresh network/rpc/bind:default
# svcprop network/rpc/bind:default | grep local_only
Once I applied these commands, the install process continued ... AT LAST!!!
Rob.

How can we achive active/active cluster setup with Oracle

Hi Experts,
How can we achieve active/active database setup with oracle WITHOUT USING RAC.. As far as I know it's impossible (unless I'm wrong)..
We are using Oracle 11.2.0.1 64bit on Windows 2008 server. We deployed Oracle FailSafe but that's more of an active/passive solution based on a windows cluster.
The other solution we were thinking about is to use DataGuard and replication.. two servers.. the oracle instance running on one server generating logs, and the other server receive the logs and apply them to the physical standby db.. Still, this is not a real active/active setup.
So, is it possible to run 2 servers in an active/active cluster and have the oracle database in an active/active setup or have the instance running on multiple nodes (at the same time)?
Thanks

Let me give you a brief explanation of what the situation is and you can be the judge..
My client have four databases with the smallest one being 20GB and the biggest around the 35gb (SGA 750mb to 1.4gb (Tiny by DB standards) and probably on a normal day, you can run all four of them on a decent desktop).. The DBs are used to keep track of people information. Through out the year, the databases are almost sitting idle, and by idle I mean, the odd update here and there, the odd report..etc. No hard real processing of any sort. Two days of the year (end of year) we have all the operators consolidating records and what's not and they will be pounding away entering data and updating the tables - with hourly reports that goes to 3rd parties. The client expects a 99.99 up time and availability during those 2 days.
Now, tell me, How can I justify using RAC and spending hundreds of thousands of dollars in licensing and what ever extra costs introduced by the complexity of the environment for the above scenario knowing that I don't have any real use for RAC for 363 days of the year; and we MIGHT need it for 2 days of the year? This is the dilemma we're facing.
Thanks
Edited by: rsar001 on Sep 3, 2010 9:42 AM

Cluster Setup hung

Similar Messages

Maybe you are looking for