Cm Manager is going down

Hi
our environment CM always going down then we have to start manually,2day morning users saying they are not able to connect and they are not able to print report
what could be the reasons

Check the CM log file for more details.
You may also run "Oracle Application Object Library Concurrent Processing Data Collection Test", check if it reports any errors/warnings.
Note: 370317.1 - Oracle Application Object Library Concurrent Processing Data Collection Test
https://metalink.oracle.com/metalink/plsql/ml2_documents.showDocument?p_database_id=NOT&p_id=370317.1

Similar Messages

EBS 11i - Concurrent Manager goes down due to network failure.

Hi,
We have a Single Node Oracle EBS 11i (11.5.10) [upgraded from 11.0.3] Production Instance on a Windows 2003 (32-Bit) server.
We have UPS support for the Server but the Netwok Switch is not on UPS. Due to this whenever there is a power trip the network goes down and due to this the Concurrent Manager (only)goes down. All other Apps Tier and DB Tier services are up inlcluding the 806 Listener (APPS_SID).
Is this how it is designed to work? Is there anyway to ensure that the Conc. Manager does not go down due to network failure.
Rgds,
Thiru

Here it is
Process monitor session ended : 02-OCT-2009 07:30:04
The Internal Concurrent Manager has encountered an error.
Review concurrent manager log file for more detailed information. : 02-OCT-2009 07:43:16 -
Shutting down Internal Concurrent Manager : 02-OCT-2009 07:45:56
Reviver is not enabled, not spawning a reviver process.
List of errors encountered:
_ 1 _
Routine AFPCMT encountered an ORACLE error. ORA-01041: internal error.
hostdef extension doesn't exist
Review your error messages for the cause of the error. (=<POINTER>)
_ 2 _
Routine AFPSMG encountered an ORACLE error. ORA-03114: not connected
to ORACLE
Review your error messages for the cause of the error. (=<POINTER>)
_ 3 _
Routine FDPCRQ encountered an ORACLE error. ORA-03113: end-of-file on
communication channel
Review your error messages for the cause of the error. (=<POINTER>)
APP-FND-01564: ORACLE error 1041 in fdudat
Cause: fdudat failed due to ORA-01041: internal error. hostdef extension doesn't exist.
The SQL statement being executed at the time of the error was: &SQLSTMT and was executed from the file &ERRFILE.
List of errors encountered:
_ 1 _
Routine AFPCAL received failure code while parsing or running your
concurrent program CPMGR
Review your concurrent request log file for more detailed information.
Make sure you are passing arguments in the correct format.
The PROPTRN_1002@PROPTRN internal concurrent manager has terminated with status 1 - giving up.

SOA managed derver is going down frequently

Hi,
We are facing one specific issue in SOA 11g environment .
SOA managed server is hosted on WebLogic Server Version: 10.3.4.0.
Server is keep on going down even after restart, after each restart it iwll come up for 15-20 minutes max and then again soa managed serve will go down.
Also in nohup logs we observed below warning :
oracle.adf.share.ADFContext> <BEA-000000>
+<Automatically initializing a DefaultContext for getCurrent.+
Caller should ensure that a DefaultContext is proper for this use.
Memory leaks and/or unexpected behaviour may occur if the automatic initialization is
performed improperly.
This message may be avoided by performing initADFContext before using getCurrent().
To see the stack trace for thread that is initializing this, set the logging level of
oracle.adf.share.ADFContext to FINEST>
Also some error messages like :
+<BEA-000000> <ServletContainerAdapter manager not initialized correctly.>+
+<BEA-000000> <Could not find saved view state for token lb617ahaj>+
Could some once please let us know what could be reason for this .
Please suggest if there is any solution for the same.
Regards,
Nupur

Hi,
We are facing one specific issue in SOA 11g environment .
SOA managed server is hosted on WebLogic Server Version: 10.3.4.0.
Server is keep on going down even after restart, after each restart it iwll come up for 15-20 minutes max and then again soa managed serve will go down.
Also in nohup logs we observed below warning :
oracle.adf.share.ADFContext> <BEA-000000>
+<Automatically initializing a DefaultContext for getCurrent.+
Caller should ensure that a DefaultContext is proper for this use.
Memory leaks and/or unexpected behaviour may occur if the automatic initialization is
performed improperly.
This message may be avoided by performing initADFContext before using getCurrent().
To see the stack trace for thread that is initializing this, set the logging level of
oracle.adf.share.ADFContext to FINEST>
Also some error messages like :
+<BEA-000000> <ServletContainerAdapter manager not initialized correctly.>+
+<BEA-000000> <Could not find saved view state for token lb617ahaj>+
Could some once please let us know what could be reason for this .
Please suggest if there is any solution for the same.
Regards,
Nupur

All Concurrent managers going down (Except Internal conc manager)

Hi All,
We are on Oracle Apps R12.0.4 and Db 10g
We have done a cloning from RAC ASM 2 node instance to Non-RAC NonASM single node.
The problem is that concurrent managers (Standard manager and other manangers) are going down automatically.
ICm obviously is trying to bring them back .
here is the log file:
Starting INVMGR Concurrent Manager : 29-JUN-2010 07:49:18
Process monitor session ended : 29-JUN-2010 07:49:18
Process monitor session started : 29-JUN-2010 07:51:18
Adding Node:(APP2), to unavailable list
Found dead process: spid=(28003), cpid=(240974), ORA pid=(44), manager=(0/4)
Found dead process: spid=(28002), cpid=(240973), ORA pid=(43), manager=(0/5)
Starting FNDSCH Concurrent Manager : 29-JUN-2010 07:51:18
Starting FNDCRM Concurrent Manager : 29-JUN-2010 07:51:18
Starting OAMCOLMGR Concurrent Manager : 29-JUN-2010 07:51:18
Starting FTE_TXN_MANAGER Concurrent Manager : 29-JUN-2010 07:51:18
Starting IEU_SH_CS Concurrent Manager : 29-JUN-2010 07:51:18
Starting IEU_WL_CS Concurrent Manager : 29-JUN-2010 07:51:18
Starting PASMGR Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting STANDARD Concurrent Manager : 29-JUN-2010 07:51:18
Starting INVMGR Concurrent Manager : 29-JUN-2010 07:51:18
Process monitor session ended : 29-JUN-2010 07:51:18
+++++++++++++++++++++++++++++++++++++++++
What i wanted to know is "Adding Node:(APP2), to unavailable list"
What does this mean and how to troubleshoot it ?
Also, i can see that STATUS column in FND_NODE table is set to "N"
Does it mean something which affects the concurrent managers?
Please help me resolve this.
Thanks,
KK

Hi,
The problem is that concurrent managers (Standard manager and other manangers) are going down automatically.
ICm obviously is trying to bring them back .Please verify the definition of the CMs and make sure none points to the RAC instances.
I would suggest you stop the CM, purge FND_NODES table and run AutoConfig on the database/application tier nodes, run cmclean.sql script, then start the CM and see if you can reproduce the issue.
Thanks,
Hussein

The Managed servers are going down frequently due to leasing renewal.

We have 3 SOA Managed Servers in our Production Environment running each of them on 3 different Machines.
The servers are going down frequently due to leasing renewal issue.
Please see the errors below. SOA_MS3 was the Cluster Leader.
From SOA_MS1 logs we see -
<Jun 15, 2011 12:11:24 AM MDT> <Warning> <Cluster> <WL-000147> <Server "SOA_MS1" failed to renew lease in the leasing basis hosted by SOA_MS3.>
<Jun 15, 2011 12:11:24 AM MDT> <Error> <Cluster> <WL-000150> <Server failed to get a connection to the leasing basis hosted by SOA_MS3 in the past 30 seconds for lease renewal. Server will shut itself down.>
<Jun 15, 2011 12:11:24 AM MDT> <Critical> <Health> <WL-310006> <Critical Subsystem ServerMigration has failed. Setting server state to FAILED.
Reason: ServerSOA_MS1 failed to renew lease in the leasing basis hosted by SOA_MS3>
<Jun 15, 2011 12:11:24 AM MDT> <Critical> <WebLogicServer> <WL-000385> <Server health failed. Reason: health of critical service 'ServerMigration' failed>
<Jun 15, 2011 12:11:24 AM MDT> <Notice> <WebLogicServer> <WL-000365> <Server state changed to FAILED>
<Jun 15, 2011 12:11:24 AM MDT> <Error> <com.bea.weblogic.kernel> <BEA-000000> <cannot load libary 'stackdump': java.lang.UnsatisfiedLinkError: no stackdump in java.library.path
>
<Jun 15, 2011 12:11:24 AM MDT> <Error> <WebLogicServer> <WL-000383> <A critical service failed. The server will shut itself down>
<Jun 15, 2011 12:11:24 AM MDT> <Notice> <WebLogicServer> <WL-000365> <Server state changed to FORCE_SHUTTING_DOWN>
<Jun 15, 2011 12:11:24 AM MDT> <Notice> <Cluster> <WL-000163> <Stopping "async" replication service>
From SOA_MS2 logs we see -
15, 2011 12:11:16 AM MDT> <Warning> <Cluster> <WL-000147> <Server "SOA_MS2" failed to renew lease in the leasing basis hosted by SOA_MS3.>
<Jun 15, 2011 12:11:16 AM MDT> <Error> <Cluster> <WL-000150> <Server failed to get a connection to the leasing basis hosted by SOA_MS3 in the past 30 seconds for lease renewal. Server will shut itself down.>
<Jun 15, 2011 12:11:16 AM MDT> <Critical> <Health> <WL-310006> <Critical Subsystem ServerMigration has failed. Setting server state to FAILED.
Reason: ServerSOA_MS2 failed to renew lease in the leasing basis hosted by SOA_MS3>
<Jun 15, 2011 12:11:16 AM MDT> <Critical> <WebLogicServer> <WL-000385> <Server health failed. Reason: health of critical service 'ServerMigration' failed>
<Jun 15, 2011 12:11:16 AM MDT> <Notice> <WebLogicServer> <WL-000365> <Server state changed to FAILED>
<Jun 15, 2011 12:11:16 AM MDT> <Error> <com.bea.weblogic.kernel> <BEA-000000> <cannot load libary 'stackdump': java.lang.UnsatisfiedLinkError: no stackdump in java.library.path
>
<Jun 15, 2011 12:11:16 AM MDT> <Error> <WebLogicServer> <WL-000383> <A critical service failed. The server will shut itself down>
<Jun 15, 2011 12:11:16 AM MDT> <Notice> <WebLogicServer> <WL-000365> <Server state changed to FORCE_SHUTTING_DOWN>
Also, In SOA_MS3 logs we see the followi error -
####<Jun 15, 2011 11:43:18 PM MDT> <Error> <Cluster> <soaprdi3> <SOA_MS3> <[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1308202998662> <WL-000168> <Failed to restart/migrate server "SOA_MS3" because of Failed to start the migratable server on one of the candidate machines
Failed to start the migratable server on one of the candidate machines
at weblogic.cluster.singleton.MigratableServerState.serverUnresponsive(MigratableServerState.java:95)
at weblogic.cluster.singleton.MigratableServersMonitorImpl.timerExpired(MigratableServersMonitorImpl.java:164)
at weblogic.timers.internal.TimerImpl.run(TimerImpl.java:273)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Thanks,

Its been many days support working on this, they do not have any clue. we escalated, not much of clue they have yet.
So, wanted put it across industry experts on SOA.
Any idea on what causes this Leasing renewal issue and the potential resolution?
Thanks

Javax.management.InstanceNotFoundException --- and the admin goes down

we are running on WLS 8.1 ,
and from past two weeks we are facing issues with the admin frequebntly going down,
i see this errros every time this goes down
javax.management.InstanceNotFoundException: mydomain:Name=ms1,Server=ms1,Type=Log (admin server:true)
at weblogic.management.internal.MBeanHomeImpl.getMBean_helper(MBeanHomeImpl.java:145)
at weblogic.management.internal.MBeanHomeImpl.getMBean(MBeanHomeImpl.java:130)
at weblogic.management.internal.AdminMBeanHomeImpl.getMBean(AdminMBeanHomeImpl.java:263)
at weblogic.management.internal.AdminMBeanHomeImpl.registerConfigMBeansForAdminChanges(AdminMBeanHomeImpl.java:1014)
at weblogic.management.internal.AdminMBeanHomeImpl_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:477)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:420)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:415)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
how ever the ms1 nolonger existis in the domain....

Can you tell more about your domain configuration, and about ms1, was it configured as part of a cluster ? Since when is it not available. Were all the servers restarted after ms1 was down.. Does your config.xml still contain reference to ms1 ? if you remove it.
The file running-managed-servers.xml also keeps track of the active managed servers, ensure that ms1 is not listed in it, and if it is there, remove the entry from this file, or delete this file before starting the admin server.

SSRS Reporting manages goes down frequently. ERROR: The underlying connection was closed: An unexpected error occurred on a send.

We have been experiencing issues with SSRS over the past 5 weeks, the service goes down frequently. And it comes up only on reboot of the server.
We had also disabled the On Access scan in Mcafee.
The service is in use for about two years, and the below error started occuring over the last few days. Please advise on this.
ui!ReportManager_0-2!24e8!08/14/2014-18:13:45:: e ERROR: The underlying connection was closed: An unexpected error occurred on a send.
ui!ReportManager_0-2!24e8!08/14/2014-18:13:45:: e ERROR: HTTP status code --> 500
-------Details--------
System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException:
An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.Socket.Send(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.ConnectStream.WriteHeaders(Boolean async)
   --- End of inner exception stack trace ---
   at Microsoft.SqlServer.ReportingServices2005.RSConnection.GetSecureMethods()
   at Microsoft.ReportingServices.UI.Global.RSWebServiceWrapper.GetSecureMethods()
   at Microsoft.SqlServer.ReportingServices2005.RSConnection.IsSecureMethod(String methodname)
   at Microsoft.SqlServer.ReportingServices2005.RSConnection.ValidateConnection()
   at Microsoft.ReportingServices.UI.Global.SecureAllAPI()
   at Microsoft.ReportingServices.UI.ReportingPage.EnsureHttpsLevel(HttpsLevel level)
   at Microsoft.ReportingServices.UI.ReportingPage.ReportingPage_Init(Object sender, EventArgs args)
   at System.EventHandler.Invoke(Object sender, EventArgs e)
   at System.Web.UI.Control.OnInit(EventArgs e)
   at System.Web.UI.Page.OnInit(EventArgs e)
   at System.Web.UI.Control.InitRecursive(Control namingContainer)
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
ui!ReportManager_0-2!24e8!08/14/2014-18:13:46:: e ERROR: Exception in ShowErrorPage: System.Threading.ThreadAbortException: Thread was being aborted.
   at System.Threading.Thread.AbortInternal()
   at System.Threading.Thread.Abort(Object stateInfo)
   at System.Web.HttpResponse.End()
   at System.Web.HttpServerUtility.Transfer(String path, Boolean preserveForm)
   at Microsoft.ReportingServices.UI.ReportingPage.ShowErrorPage(String errMsg) at    at System.Threading.Thread.AbortInternal()
   at System.Threading.Thread.Abort(Object stateInfo)
   at System.Web.HttpResponse.End()
   at System.Web.HttpServerUtility.Transfer(String path, Boolean preserveForm)
   at Microsoft.ReportingServices.UI.ReportingPage.ShowErrorPage(String errMsg)
library!ReportServer_0-1!1c90!08/14/2014-18:13:51:: Call to ListParentsAction(/Data Sources).
library!ReportServer_0-1!1c90!08/14/2014-18:13:51:: Call to ListParentsAction(/Data Sources).
library!ReportServer_0-1!1c90!08/14/2014-18:13:51:: Call to ListChildrenAction(/Data Sources, False).
webserver!ReportServer_0-1!1c90!08/14/2014-18:13:51:: i INFO: Processed folder '/Data Sources'
library!ReportServer_0-1!68c!08/14/2014-18:13:56:: Call to GetItemTypeAction(/).
library!ReportServer_0-1!68c!08/14/2014-18:13:56:: Call to ListParentsAction(/).
library!ReportServer_0-1!68c!08/14/2014-18:13:56:: Call to ListChildrenAction(/, False).
webserver!ReportServer_0-1!68c!08/14/2014-18:13:56:: i INFO: Processed folder '/'

Hi SThiyag,
Based on your error message, it may be related to the “SecureConnectionLevel” in the RSReportServer.config file. Please go to C:\Program Files\Microsoft SQL Server\MSRS11.MSSQLSERVER\Reporting Services\ReportServer, use Notepad to open the RSReportServer.config,
modify <Add Key="SecureConnectionLevel" Value="0"/>, change the value into "1" or "2".
(Note: Please backup the file before modify it.)
Acceptable values are:
3 Most secure—Use SSL for absolutely everything.
2 Secure—Use SSL for rendering and methods that pass credentials but don't insist on it for all SOAP calls.
1 Basic Security—Accepts http but rejects any calls that might be involved in the passing of credentials.
0 Least Secure—Don't use SSL at all
Here is a blog for error message "The underlying connection was closed...", please refer to the link below:
http://blogs.msdn.com/b/engelsr/archive/2005/11/29/the-underlying-connection-was-closed.aspx
If you have any question, please feel free to ask.
Best Regards,
Simon Hou

What happens to client session when host goes down?

I have a windows 2012 R2 RDP environment. 1 Broker and 3 Session Host Servers.
My question is as follows:
User connects via broker to one of the Host servers. Host server goes down, while user is still connected. Session is written in Broker database. User wants to connect again, but broker sends him on to the server which is down and so he cannot connect.
Other situation is similar, User connects to host, administrator drains host, stopping new connections. User disconnects session, session is still in Broker, when user tries to reconnect is sent to drained server but cannot login, as he is considered new
connection.
What am I missing?
Aaron

Hi Aaron,
Thank you for posting in Windows Server Forum.
We need to configure RDSH in a load balance farm where it can be managed by RD Connection Broker. For that we need to use DNS RR where need to make entry for all RDSH server which will be handled by RDCB Server or we can use any load balancing method to load
balance the different RDSH server and thus that connection will allow the disconnected session for reconnection and can identify the connection.
The RD Connection Broker for RDS serves the following main purposes:
• Determining the best RD Session Host server to connect to - this is based on the amount of current sessions per RD Session Host Server as a form of load balancing.
• Reconnecting a user to an existing session, in case there is one - this is called reconnecting to disconnected sessions.
• Being a resource for RD Web Access to deliver information about what Remote Apps are available on RD Session Host Servers that are a member of the broker.
More information.
Remote Desktop Server farms explained (Part 1
& Part 2) (Link is for Server 2008 R2 but it’s only for reference).
Step by Step Windows 2012 R2 Remote Desktop Services – Part 3 (For Server 2012 R2)
Hope it helps!
Thanks.
Dharmesh Solanki

OAS 4.0.8.1 for Linux: wrksf goes down

After having installed the OAS 4.0.8.1 for
Linux, I have a problem getting it to work.
I use the OAS Manager to start the following
four processes :
oassrv "Oracle Application Server"
wrksf "Cartridge Server Factory"
otsfacsrv "Transaction Server Factory"
orb "ORB Servers"
The four processes come up, but after a
minute or so the 'wrksf' process goes
down.
After restarting the wrksf in
command line mode (using the owsctl utility)
the same occurs: it stays up for about a
minute or so, and then goes down.
The log files don't show any error messages.
Does anybody have a clue in which direction
I should search ?
Can any tracing be turned on ?

After I fixed some of the things I broke while searching for the wrksf failure, the PL/SQL Cartridge now works for me too. And, after failing to get the DB Browser to work by loading it into the SCOTT schema, I got it to work by loading it into SYS. (SCOTT can't see the DBA_* views so loading the DB Browser into the database failed.) I haven't tried any Java Servlets yet, but that's next on my list.
If you want to try my wrksf workaround, rather than using Christoph's, here's how you can do it.
1. Backup liborb.so in the $ORACLE_HOME/orb/4.0/lib directory, just in case something goes wrong.
2. Using a hex editor, such as emacs hexl-find-file or ghex, to edit liborb.so, find the string "/proc/stat".
3. Change the directory, "/proc" to something like "/pfoo" and write the shared library file back to liborb.so. You now have the required modified liborb.so.
4. Make the /pfoo directory and cp /proc/stat /pfoo.
5. Edit /pfoo/stat and duplicate the first line, which should be the cpu line. This will let the metrics parser handle it. Make sure that /pfoo/stat has read access.
Of course, the values that the metrics code gets are now bogus, but that does not seem to be a fatal problem.
Hopefully somebody from Oracle is reading this and the real fix will be in the mail soon.

Cannot keep Server up and running. Shutdown requested by OVM. The system is going down for system halt NOW!

I have OVS 3.x.x and OVM 3.x.x running on the same physical server. I had some issues trying to configure storage via OVM and I issued a shutdown (I think) from the OVM Web Interface. Now I am unable to keep the server up and running. Everytime I turn on the server, after Oracle VM Manager starts, it shuts down the server automatically within a minute or two after startup.
I am able to login to the server via Putty and stop ovmm service manually. When Oracle VM Manager service is stopped, (SERVICE OVMM STOP) the server does not shutdown automatically. However as soon as I restart the OVMM service, it shuts down the server with message...
Shutdown requested by OVM. The system is going down for system halt NOW!
How do I find out what is causing OVM to issue the shutdown command and edit and fix via command line?

Since your server was continually rebooting, do you remember if you issued a shutdown or reboot from OVM web interface? Mine was just shutting down. Have you tried to use the Oracle Virtual Manage Web Interface to either reboot or shutdown the server after your upgrade to 3.2.4? FYI I do not have support. This is my sandbox.
user5728486 wrote:
After a week or so without a resolution from support, I noticed ....
Are you saying that even paying for Oracle Support, they did not resolve this issue?

Server Goes Down Alert

Hello,
I am running SCOM 2012 R2 and I am trying to setup an alert to let me know when a server goes down. I tried creating a rule that monitored the Management Server's "Operations Manager" Event Log for ID 20022. I also configured
the rule that parameter #2 must have my servers name. I also tried EventDescription conatains my servers name. None of them worked. But I looked in the Event Log of my Management Server and as I looked at all of the 20022 entries, I noticed
the computer I was on was listed as not heartbeating. Two of my other servers that are currently on were listed as not heartbeating. So obviously monitoring this 20022 entry is not reliable. I do not want to setup a monitor for something
like Ping Status. I want to try to avoid using monitors because they could be taxing on the servers, my management server, and my network.

Hi
yes event id 20022 is not reliable and never go for windows event id monitor for server shutdown or unavailability on agent , as scom health service will be stopped or connection closed with MS at the time of shutdown on agent.
you need to go for ICMP port monitoring for server availability, as port monitoring run from remote server(watcher node) and can capture server unavailability and ping test.
refer below link for port monitoring
http://technet.microsoft.com/en-us/library/hh457544.aspx
http://www.systemcentercentral.com/custom-tcp-port-monitoring-management-pack-for-scom-2012/
regards
sridhar v

Oracle 10g Agent goes down itself

I have been observing the Oracle 10g agent goes down by itself and in grid control it reports as Agent unreachable. where do I see the log why Agent is going down on target everytime.

I guess the location is D:\oracle\agent10g\sysman\log
However, I do not see any information why th agent went down ..
log looks like below
2009-12-29 15:35:40,552 Thread-5812 [IBM WebSphere Application Server] InstanceProperty (version) is marked OPTIONAL but is being used
(00506)
2009-12-29 15:35:47,599 Thread-5812 EMAgent started successfully
(00702)
2010-01-06 11:00:36,085 Thread-4360 Starting Agent 10.2.0.4.0 from D:\oracle\agent10g
(00701)
2010-01-06 11:00:37,929 Thread-4360 [E-Mail IMAP Server] InstanceProperty (imap_host) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:38,804 Thread-4360 Invalid expression (CacheFlushSuccess __delta) > 0 ? (CacheFlushSuccess __delta ) : 0
(00101)
2010-01-06 11:00:39,132 Thread-4360 [Database Instance] InstanceProperty (DBVersion) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:39,445 Thread-4360 [Identity Manager Repository] InstanceProperty (Port) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:39,445 Thread-4360 [Identity Manager Repository] InstanceProperty (SID) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:39,445 Thread-4360 [Identity Manager Repository] InstanceProperty (url) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:39,445 Thread-4360 [Identity Manager Repository] InstanceProperty (jdbcdriver) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:39,585 Thread-4360 [Oracle BPEL Process Manager] InstanceProperty (oc4jTargetName) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:40,616 Thread-4360 [E-Mail POP Server] InstanceProperty (pop_host) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:40,726 Thread-4360 [Cluster Database] InstanceProperty (DBVersion) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:40,851 Thread-4360 [Siebel Gateway Server] InstanceProperty (SIEBEL_GATEWAY) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:40,851 Thread-4360 [Siebel Gateway Server] InstanceProperty (SIEBEL_PORT) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:40,866 Thread-4360 [Siebel Server] InstanceProperty (filesystem:filesystem) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:40,866 Thread-4360 [Siebel Server] InstanceProperty (filesystem:installdir) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:40,866 Thread-4360 [Siebel Server] InstanceProperty (filesystem:logdir) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:40,898 Thread-4360 [Load Balancer Switch] InstanceProperty (snmpTimeout) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:40,960 Thread-4360 [E-Mail SMTP Inbound Server] InstanceProperty (smtp_host) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:41,007 Thread-4360 [E-Mail SMTP Outbound Server] InstanceProperty (smtp_host) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:41,085 Thread-4360 Undefined column name interval in expression ( active.requests > 0) ? active.requests/(interval*60) : 0
(00104)
2010-01-06 11:00:41,132 Thread-4360 [BEA WebLogic Managed Server] InstanceProperty (version) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:41,226 Thread-4360 [IBM WebSphere Application Server] InstanceProperty (version) is marked OPTIONAL but is being used
(00506)
2010-01-06 11:00:48,054 Thread-4360 EMAgent started successfully (00702)

AP 1200 Just goes down hard

We have a 1200 Aironet just go down hard for no reason through out the day, we replaced the Power Injector with a C2960 POE Switch, we also replaced the cabiling. When it goes down hard all three lights are still green as if the AP is still up. We verified the IOS with other stores and everything is the same on the configurations, versions, and IOS.
Can anyone give us a suggestion on what the issue or something else we can look at that we might be over looking?
Couple more things that we have done:
We verified the IOS with other stores and everything is the same on the configurations, versions, and IOS.
Thanks in advance......

Hello John,
As per your query i can suggest you the following solution-
Please check the POE for the AP and whether it is proper or not.
Types of Power over Ethernet
This section describes various types of Power over Ethernet (PoE) including Cisco pre-standard power, IEEE 802.3af power, and IEEE 802.3at power.
Cisco Pre-Standard Power
The first generation of Cisco pre-standard PoE was designed to power devices such as Cisco IP phones and access points. This pre-standard solution had relatively low power requirements (approximately 6 to 7 watts). Later generations of pre-standard power supported higher power modes (up to 15 Watts) and added power negotiation via Cisco Discovery Protocol (CDP). In July 2003, the IEEE ratified the 802.1af standard (up to 15.4 watts of power). With the ratification of IEEE 802.3af, Cisco supports both IEEE 802.3af and Cisco pre-standard PoE concurrently. Cisco has also extended pre-standard power management using CDP negotiation to Cisco IEEE 802.3af compliant devices to further optimize Power Source Equipment (PSE) power management.
When using a Cisco pre-standard source for power, it is important to check the power draw in watts (Table 1) and verify that the PSE can supply enough wattage to the powered device. Depending on the version of your PSE, enough power may not be available to supply some of the newer access points.
IEEE 802.3af Power
The IEEE 802.af-2003 PoE standard:
•Defines terminology to describe a port that acts as a power source to a powered device.
•Defines how a powered device is detected.
•Defines two methods of delivering PoE to the discovered device requiring power.
•End-Point PSE—PoE-capable Ethernet port. Power may be delivered by an end-point PSE to a powered device using the active data wires of an Ethernet port or the spare wires. An end-point PSE, such as a PoE-capable Ethernet switch may implement either scheme.
•Mid-span PSE—Can be used to deliver PoE if an existing non-PoE-capable Ethernet switch is used. If a mid-span PSE is used, it can only implement power delivery over the spare pairs of the copper cabling and cannot be used to deliver PoE over 1000BASE-T connections.
Hope this will help you.

Host server live migration causing Guest Cluster node goes down

Hi
I have two node Hyper host cluster , Im using converged network for Host management,Live migartion and cluster network. And Separate NICs for ISCSI multi-pathing. When I live migrate the Guest node from one host to another , within guest cluster the node
is going down. I have increased clusterthroshold and clusterdelay values. Guest nodes are connecting to ISCSI network directly from ISCSI initiator on Server 2012.
The converged networks for management ,cluster and live migration networks are built on top of a NIC Team with switch Independent mode and load balancing as Hyper V port.
I have VMQ enabled on Converged fabric and jumbo frames enabled on ISCSI.
Can Anyone guess why would live migration cause failure on the guest node.
thanks
mumtaz

Repost here: http://social.technet.microsoft.com/Forums/en-US/winserverhyperv/threads
in the Hyper-V forum. You'll get a lot more help there.
This forum is for Virtual Server 2005.

OSB dashboard Service Health tab broken when a managed server is down

I have shut down one of the managed servers in my three-member OSB 10.3.1 cluster to load test the reduced configuration, and that has broken the "service health" tab on the OSB console dashboard.
The problem is that with the mgd1 server shut down (the one the Singleton marker ear is deployed to, if that matters), in the OSB console's Dashboard everything in the "Service Health" tab is broken. I get the warning message "Unable to obtain metrics data from the server" at the top of the page whenever I go to the dashboard. I can't even look at the displays for the individual (mgd2, mgd3) servers that are running. I get the following error in the log when trying to access the service health for the services in the cluster, or even if I try to get the health of services in either of the two remaining cluster members individually:
+####<Apr 8, 2010 11:00:02 PM EDT> <Error> <AlertLog> <nj09mhm0033.edmz.mcgraw-hill.com> <SS_ESB_Admin> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1270782002573> <BEA-000000> <WLDFAccessRuntimeMBean is not available for server SS_ESB_mgd1>+
Also, as part of the load test procedure, I need to collect performance stats from the proxy service pipeline and the business service. To do this, I've written JMX code called out of soapUI scripts to reset the stats on these services before the run, then collect the stats after the run ends; the code connects to the admin server. This too is broken when the mgd1 server is shut down.
I get the error below when trying to collect the cluster stats from the Admin server from the JMX code. This error is fatal, and no statistics come back from the rest of the cluster either. (you can ignore all the groovy stuff in the stack trace, it's the nested Exceptions that are more interesting).
Does anyone have an idea about what's going on, and how I fix this? I would have thought the DomainService Runtime MBean would be able to catch and handle an Exception like this if one of the managed servers were down, and collect the stats from the available part of the cluster, but it looks like that's not the case.
TIA for any help.
/Steve Elkind
java.lang.reflect.UndeclaredThrowableException
at $Proxy6.getBusinessServiceStatistics(Unknown Source)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever.getAndResetStatsForMonitoredBizService(ServiceStatsRetriever.java:193)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever.getStatsForMonitoredBizService(ServiceStatsRetriever.java:142)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever$GetAndResetStatisticsTask.run(ServiceStatsRetriever.java:652)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever$GetAndResetStatisticsTask.getStats(ServiceStatsRetriever.java:619)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever.getStats(ServiceStatsRetriever.java:779)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever$getStats.call(Unknown Source)
at Script1.sleepGetStats(Script1.groovy:75)
at Script1$sleepGetStats.callCurrent(Unknown Source)
at Script1.runBenchMarkTest(Script1.groovy:104)
at Script1$runBenchMarkTest.callCurrent(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:44)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:143)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:163)
at Script1.run(Script1.groovy:152)
at com.eviware.soapui.support.scripting.groovy.SoapUIGroovyScriptEngine.run(SoapUIGroovyScriptEngine.java:93)
at com.eviware.soapui.impl.wsdl.loadtest.WsdlLoadTest.runSetupScript(WsdlLoadTest.java:915)
at com.eviware.soapui.impl.wsdl.loadtest.WsdlLoadTestRunner.start(WsdlLoadTestRunner.java:102)
at com.eviware.soapui.impl.wsdl.loadtest.WsdlLoadTest.run(WsdlLoadTest.java:418)
at com.eviware.soapui.impl.wsdl.panels.loadtest.WsdlLoadTestDesktopPanel$RunLoadTestAction.actionPerformed(WsdlLoadTestDesktopPanel.java:424)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1995)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2318)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:272)
at java.awt.Component.processMouseEvent(Component.java:6263)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3267)
at java.awt.Component.processEvent(Component.java:6028)
at java.awt.Container.processEvent(Container.java:2041)
at java.awt.Component.dispatchEventImpl(Component.java:4630)
at java.awt.Container.dispatchEventImpl(Container.java:2099)
at java.awt.Component.dispatchEvent(Component.java:4460)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4574)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4238)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4168)
at java.awt.Container.dispatchEventImpl(Container.java:2085)
at java.awt.Window.dispatchEventImpl(Window.java:2478)
at java.awt.Component.dispatchEvent(Component.java:4460)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever$ServiceDomainMBeanInvocationHandler.invoke(ServiceStatsRetriever.java:567)
... 46 more
Caused by: com.bea.wli.monitoring.MonitoringException: [Monitoring Runtime:473057]Failed to get statistics for services due to java.rmi.UnknownHostException: Could not discover administration URL for server 'SS_ESB_mgd1'
at com.bea.wli.sb.management.monitoring.ServiceDomainMBeanImpl.getStatisticsForServices(ServiceDomainMBeanImpl.java:967)
at com.bea.wli.sb.management.monitoring.ServiceDomainMBeanImpl.getServiceStatistics(ServiceDomainMBeanImpl.java:802)
at com.bea.wli.sb.management.monitoring.ServiceDomainMBeanImpl.getBusinessServiceStatistics(ServiceDomainMBeanImpl.java:366)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at weblogic.management.jmx.modelmbean.WLSModelMBean.invoke(WLSModelMBean.java:437)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
at weblogic.management.mbeanservers.domainruntime.internal.FederatedMBeanServerInterceptor.invoke(FederatedMBeanServerInterceptor.java:255)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$16.run(WLSMBeanServerInterceptorBase.java:447)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:445)
at weblogic.management.mbeanservers.internal.SecurityMBeanMgmtOpsInterceptor.invoke(SecurityMBeanMgmtOpsInterceptor.java:65)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$16.run(WLSMBeanServerInterceptorBase.java:447)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:445)
at weblogic.management.mbeanservers.internal.SecurityInterceptor.invoke(SecurityInterceptor.java:443)
at weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$10$1.run(AuthenticatedSubjectInterceptor.java:582)
at weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$10.run(AuthenticatedSubjectInterceptor.java:580)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor.invoke(AuthenticatedSubjectInterceptor.java:573)
at weblogic.management.jmx.mbeanserver.WLSMBeanServer.invoke(WLSMBeanServer.java:307)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1366)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
at javax.management.remote.rmi.RMIConnectionImpl_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:589)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:473)
at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Caused by: java.rmi.UnknownHostException: Could not discover administration URL for server 'SS_ESB_mgd1'
at weblogic.protocol.URLManager.findAdministrationURL(URLManager.java:175)
at com.bea.wli.sb.util.server.AggregatorUtils.getAggregatorServerAdminURL(AggregatorUtils.java:105)
at com.bea.wli.sb.management.monitoring.ServiceDomainMBeanImpl.getStatisticsRetrieverMBean(ServiceDomainMBeanImpl.java:2036)
at com.bea.wli.sb.management.monitoring.ServiceDomainMBeanImpl.getStatisticsForServices(ServiceDomainMBeanImpl.java:933)
... 35 more
java.lang.reflect.UndeclaredThrowableException
at $Proxy6.getBusinessServiceStatistics(Unknown Source)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever.getAndResetStatsForMonitoredBizService(ServiceStatsRetriever.java:193)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever.getStatsForMonitoredBizService(ServiceStatsRetriever.java:142)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever$GetAndResetStatisticsTask.run(ServiceStatsRetriever.java:652)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever$GetAndResetStatisticsTask.getStats(ServiceStatsRetriever.java:619)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever.getStats(ServiceStatsRetriever.java:779)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever$getStats.call(Unknown Source)
at Script1.sleepGetStats(Script1.groovy:75)
at Script1$sleepGetStats.callCurrent(Unknown Source)
at Script1.runBenchMarkTest(Script1.groovy:104)
at Script1$runBenchMarkTest.callCurrent(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:44)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:143)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:163)
at Script1.run(Script1.groovy:152)
at com.eviware.soapui.support.scripting.groovy.SoapUIGroovyScriptEngine.run(SoapUIGroovyScriptEngine.java:93)
at com.eviware.soapui.impl.wsdl.loadtest.WsdlLoadTest.runSetupScript(WsdlLoadTest.java:915)
at com.eviware.soapui.impl.wsdl.loadtest.WsdlLoadTestRunner.start(WsdlLoadTestRunner.java:102)
at com.eviware.soapui.impl.wsdl.loadtest.WsdlLoadTest.run(WsdlLoadTest.java:418)
at com.eviware.soapui.impl.wsdl.panels.loadtest.WsdlLoadTestDesktopPanel$RunLoadTestAction.actionPerformed(WsdlLoadTestDesktopPanel.java:424)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1995)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2318)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:272)
at java.awt.Component.processMouseEvent(Component.java:6263)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3267)
at java.awt.Component.processEvent(Component.java:6028)
at java.awt.Container.processEvent(Container.java:2041)
at java.awt.Component.dispatchEventImpl(Component.java:4630)
at java.awt.Container.dispatchEventImpl(Container.java:2099)
at java.awt.Component.dispatchEvent(Component.java:4460)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4574)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4238)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4168)
at java.awt.Container.dispatchEventImpl(Container.java:2085)
at java.awt.Window.dispatchEventImpl(Window.java:2478)
at java.awt.Component.dispatchEvent(Component.java:4460)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.mgh.im.esb.jmxutils.ServiceStatsRetriever$ServiceDomainMBeanInvocationHandler.invoke(ServiceStatsRetriever.java:567)
... 46 more
Caused by: com.bea.wli.monitoring.MonitoringException: [Monitoring Runtime:473057]Failed to get statistics for services due to java.rmi.UnknownHostException: Could not discover administration URL for server 'SS_ESB_mgd1'
at com.bea.wli.sb.management.monitoring.ServiceDomainMBeanImpl.getStatisticsForServices(ServiceDomainMBeanImpl.java:967)
at com.bea.wli.sb.management.monitoring.ServiceDomainMBeanImpl.getServiceStatistics(ServiceDomainMBeanImpl.java:802)
at com.bea.wli.sb.management.monitoring.ServiceDomainMBeanImpl.getBusinessServiceStatistics(ServiceDomainMBeanImpl.java:366)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at weblogic.management.jmx.modelmbean.WLSModelMBean.invoke(WLSModelMBean.java:437)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
at weblogic.management.mbeanservers.domainruntime.internal.FederatedMBeanServerInterceptor.invoke(FederatedMBeanServerInterceptor.java:255)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$16.run(WLSMBeanServerInterceptorBase.java:447)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:445)
at weblogic.management.mbeanservers.internal.SecurityMBeanMgmtOpsInterceptor.invoke(SecurityMBeanMgmtOpsInterceptor.java:65)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$16.run(WLSMBeanServerInterceptorBase.java:447)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:445)
at weblogic.management.mbeanservers.internal.SecurityInterceptor.invoke(SecurityInterceptor.java:443)
at weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$10$1.run(AuthenticatedSubjectInterceptor.java:582)
at weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$10.run(AuthenticatedSubjectInterceptor.java:580)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor.invoke(AuthenticatedSubjectInterceptor.java:573)
at weblogic.management.jmx.mbeanserver.WLSMBeanServer.invoke(WLSMBeanServer.java:307)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1366)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
at javax.management.remote.rmi.RMIConnectionImpl_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:589)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:473)
at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Caused by: java.rmi.UnknownHostException: Could not discover administration URL for server 'SS_ESB_mgd1'
at weblogic.protocol.URLManager.findAdministrationURL(URLManager.java:175)
at com.bea.wli.sb.util.server.AggregatorUtils.getAggregatorServerAdminURL(AggregatorUtils.java:105)
at com.bea.wli.sb.management.monitoring.ServiceDomainMBeanImpl.getStatisticsRetrieverMBean(ServiceDomainMBeanImpl.java:2036)
at com.bea.wli.sb.management.monitoring.ServiceDomainMBeanImpl.getStatisticsForServices(ServiceDomainMBeanImpl.java:933)
... 35 more

As a follow-up, does anyone know if I can simply retarget the two OSB Singleton Marker Apps (shown below from config.xml) to another server instance to fix this and any other Singleton issues (e.g., flie transport, ftp, and email pollers), or is it more involved than that?
<app-deployment>
<name>ALSB Cluster Singleton Marker Application</name>
<target>SS_ESB_mgd1</target>
<module-type>ear</module-type>
<source-path>/opt/bea/osb1031/osb_10.3/lib/common/clustersingletonmarker.ear</source-path>
<deployment-order>80</deployment-order>
<security-dd-model>DDOnly</security-dd-model>
</app-deployment>
<app-deployment>
<name>ALSB Domain Singleton Marker Application</name>
<target>SS_ESB_mgd1</target>
<module-type>ear</module-type>
<source-path>/opt/bea/osb1031/osb_10.3/lib/common/domainsingletonmarker.ear</source-path>
<deployment-order>85</deployment-order>
<security-dd-model>DDOnly</security-dd-model>
</app-deployment>

Cm Manager is going down

Similar Messages

Maybe you are looking for