Programmatically starting a cluster?

All,
I've been using Coherence for many years and am now experimenting with Extend in the new 12.1.3 release.  I'm trying to run my storage nodes as a simple webapp in Tomcat.  My idea for starting to cache node was to use a ServletContextListener.  I would start the cache in contextInitialized() and stop it in contextDestroyed().
I can't get the startup to work correctly, however.  I need to use a method that doesn't block, so simply calling DefaultCacheServer.main() won't do.
The documentation at 8 Starting and Stopping Cluster Members (12c Release 2 (12.1.2)) is just plain wrong.  For example, there is no constructor to ExtensibleConfigurableCacheFactory that takes a single String as input.
Browsing the API, I see I'm supposed to do something like this:
ConfigurableCacheFactory cacheFactory =
CacheFactory.getCacheFactoryBuilder().getConfigurableCacheFactory(
"coherence-server-config.xml",
Thread.currentThread().getContextClassLoader());
DefaultCacheServer dcs = new DefaultCacheServer(cacheFactory);
dcs.startDaemon(5000);
At first this appears to work.  I get to "Started DefaultCacheServer..." and the output shows that it used the config file I told it to.
My config file has 1 cache scheme and 1 proxy scheme.  Both have autostart=true.
However when I try put() to the cache from an Extend client, I get "An exception occurred while processing a EnsureCacheRequest for Service=Proxy:ExtendTcpProxyService:TcpAcceptor: java.lang.IllegalStateException: Service "DistributedCache" has been started by a different configurable cache factory."  I guess that is telling me that those 2 services were started by different configs.  I didn't expect that.  Do I need to do something else to start start the services in addition to the cache service?
FWIW, these are all of the "loaded" lines in my output:
2014-09-08 09:12:09.295/0.615 Oracle Coherence 12.1.3.0.0 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/tangosol-coherence.xml"
2014-09-08 09:12:09.406/0.726 Oracle Coherence 12.1.3.0.0 <Info> (thread=main, member=n/a): Loaded operational overrides from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/tangosol-coherence-override-dev.xml"
2014-09-08 09:12:09.802/1.122 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=main, member=n/a): Loaded cache configuration from "file:/C:/projects/jgregg/coherence-extend-server/target/classes/META-INF/coherence-server-config.xml"
2014-09-08 09:12:14.819/6.139 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=Cluster, member=1): Loaded POF configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/pof-config.xml"
2014-09-08 09:12:14.858/6.178 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=Cluster, member=1): Loaded included POF configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/coherence-pof-config.xml"
2014-09-08 09:12:14.975/6.295 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=main, member=1): Loaded Reporter configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/reports/report-group.xml"
2014-09-08 09:23:58.297/709.617 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=Proxy:ExtendTcpProxyService:TcpAcceptorWorker:0, member=1): Loaded cache configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/coherence-cache-config.xml"
2014-09-08 09:23:58.361/709.681 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=Proxy:ExtendTcpProxyService:TcpAcceptorWorker:0, member=1): Loaded cache configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/internal-txn-cache-config.xml"
As you can see, lines 3 and 7 are for different config files (and different threads.)  Line 7 and 8 only occur after an Extend client tries to do something.
Finally, my config works fine as long as I start the server from the command line.
How am I supposed to do this?
thanks
John

All,
I've been using Coherence for many years and am now experimenting with Extend in the new 12.1.3 release.  I'm trying to run my storage nodes as a simple webapp in Tomcat.  My idea for starting to cache node was to use a ServletContextListener.  I would start the cache in contextInitialized() and stop it in contextDestroyed().
I can't get the startup to work correctly, however.  I need to use a method that doesn't block, so simply calling DefaultCacheServer.main() won't do.
The documentation at 8 Starting and Stopping Cluster Members (12c Release 2 (12.1.2)) is just plain wrong.  For example, there is no constructor to ExtensibleConfigurableCacheFactory that takes a single String as input.
Browsing the API, I see I'm supposed to do something like this:
ConfigurableCacheFactory cacheFactory =
CacheFactory.getCacheFactoryBuilder().getConfigurableCacheFactory(
"coherence-server-config.xml",
Thread.currentThread().getContextClassLoader());
DefaultCacheServer dcs = new DefaultCacheServer(cacheFactory);
dcs.startDaemon(5000);
At first this appears to work.  I get to "Started DefaultCacheServer..." and the output shows that it used the config file I told it to.
My config file has 1 cache scheme and 1 proxy scheme.  Both have autostart=true.
However when I try put() to the cache from an Extend client, I get "An exception occurred while processing a EnsureCacheRequest for Service=Proxy:ExtendTcpProxyService:TcpAcceptor: java.lang.IllegalStateException: Service "DistributedCache" has been started by a different configurable cache factory."  I guess that is telling me that those 2 services were started by different configs.  I didn't expect that.  Do I need to do something else to start start the services in addition to the cache service?
FWIW, these are all of the "loaded" lines in my output:
2014-09-08 09:12:09.295/0.615 Oracle Coherence 12.1.3.0.0 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/tangosol-coherence.xml"
2014-09-08 09:12:09.406/0.726 Oracle Coherence 12.1.3.0.0 <Info> (thread=main, member=n/a): Loaded operational overrides from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/tangosol-coherence-override-dev.xml"
2014-09-08 09:12:09.802/1.122 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=main, member=n/a): Loaded cache configuration from "file:/C:/projects/jgregg/coherence-extend-server/target/classes/META-INF/coherence-server-config.xml"
2014-09-08 09:12:14.819/6.139 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=Cluster, member=1): Loaded POF configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/pof-config.xml"
2014-09-08 09:12:14.858/6.178 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=Cluster, member=1): Loaded included POF configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/coherence-pof-config.xml"
2014-09-08 09:12:14.975/6.295 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=main, member=1): Loaded Reporter configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/reports/report-group.xml"
2014-09-08 09:23:58.297/709.617 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=Proxy:ExtendTcpProxyService:TcpAcceptorWorker:0, member=1): Loaded cache configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/coherence-cache-config.xml"
2014-09-08 09:23:58.361/709.681 Oracle Coherence GE 12.1.3.0.0 <Info> (thread=Proxy:ExtendTcpProxyService:TcpAcceptorWorker:0, member=1): Loaded cache configuration from "jar:file:/C:/Users/John.E.Gregg/.m2/repository/com/oracle/coherence/coherence/12.1.3-0-0/coherence-12.1.3-0-0.jar!/internal-txn-cache-config.xml"
As you can see, lines 3 and 7 are for different config files (and different threads.)  Line 7 and 8 only occur after an Extend client tries to do something.
Finally, my config works fine as long as I start the server from the command line.
How am I supposed to do this?
thanks
John

Similar Messages

  • Having issue with start weblogic cluster with tangosol cluster

    Hi,
    Oracle Coherence Version 3.3.1/389p1
    Grid Edition: Development mode
    We are using Weblogic 8.1.5 with Tangosol 3.3.1 on Linux servers.
    And we added the initializing logic in the servlet's init() method to get all NamedCaches and put into the ServletContext.
    When we start weblogic cluster, the first weblogic member will startup successfully with following messages :
    <Nov 7, 2007 10:12:30 AM EST> <Info> <HTTP> <BEA-101047> <[2007-11-07 10ServletContext(id=259640596,name=clusterqa,context-path=)] initObjects: init>:12:31.565 Oracle Coherence 3.3.1/389p1 <Info> (thread=Main Thread, member=n/a): Loaded operational configuration from resource "zip:/home/server/clusterqa/wls81/DOCVIEW/docqa1/.wlnotdelete/extract/docqa1_DOC_clusterqa/jarfiles/WEB-INF/lib/coherence.jar!/tangosol-coherence.xml"
    2007-11-07 10:12:31.598 Oracle Coherence 3.3.1/389p1 <Info> (thread=Main Thread, member=n/a): Loaded operational overrides from file "/home/www/WEB-INF/lib/tangosol-coherence-override.xml"
    Oracle Coherence Version 3.3.1/389p1
    Grid Edition: Development mode
    Copyright (c) 2000-2007 Oracle. All rights reserved.
    2007-11-07 10:12:31.938 Oracle Coherence GE 3.3.1/389p1 <Info> (thread=Main Thread, member=n/a): Loaded cache configuration from file "/home/www/WEB-INF/lib/pub-search-cache-config.xml"
    2007-11-07 10:12:31.983 Oracle Coherence GE 3.3.1/389p1 <Info> (thread=Main Thread, member=n/a): sun.misc.AtomicLong is not supported on this JVM; using a synchronized counter. Though safe to ignore, you may upgrade to BEA's 1.5 JVM to fix this issue.
    2007-11-07 10:12:33.267 Oracle Coherence GE 3.3.1/389p1 <Warning> (thread=Main Thread, member=n/a): UnicastUdpSocket failed to set receive buffer size to 1428 packets (2096304 bytes); actual size is 89 packets (131071 bytes). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
    2007-11-07 10:12:34.118 Oracle Coherence GE 3.3.1/389p1 <D5> (thread=Cluster, member=n/a): Service Cluster joined the cluster with senior service member n/a
    2007-11-07 10:12:37.508 Oracle Coherence GE 3.3.1/389p1 <Info> (thread=Cluster, member=n/a): Created a new cluster with Member(Id=1, Timestamp=2007-11-07 10:12:33.323, Address=10.5.176.86:8088, MachineId=48982, Edition=Grid Edition, Mode=Development, CpuCount=4, SocketCount=2) UID=0x0A05B056000001161AAB782BBF561F98
    2007-11-07 10:12:37.736 Oracle Coherence GE 3.3.1/389p1 <D5> (thread=Invocation:Management, member=1): Service Management joined the cluster with senior service member 1
    2007-11-07 10:12:38.168 Oracle Coherence GE 3.3.1/389p1 <D5> (thread=DistributedCache, member=1): Service DistributedCache joined the cluster with senior service member 1
    <Nov 7, 2007 10:12:38 AM EST> <Info> <HTTP> <BEA-101047> <[ServletContext(id=259640596,name=clusterqa,context-path=)] xslProcessor: init>
    But trying to start the second weblogic member server, the startup process is stucked after tangosol cache initialization and the second weblogic member server never up running. Please see following messages :
    <Nov 7, 2007 9:49:38 AM EST> <Info> <HTTP> <BEA-101047> <[ServletContext(id=153019550,name=clusterqa,context-path=)] initDSNames: init>
    <Nov 7, 2007 9:49:42 AM EST> <Info> <HTTP> <BEA-101047> <[ServletContext(id=153019550,name=clusterqa,context-path=)] initObjects: init>
    2007-11-07 09:49:43.156 Oracle Coherence 3.3.1/389p1 <Info> (thread=Main Thread, member=n/a): Loaded operational configuration from resource "zip:/home/server/clusterqa/wls81/DOCVIEW/docqa2/.wlnotdelete/extract/docqa2_DOC_clusterqa/jarfiles/WEB-INF/lib/coherence.jar!/tangosol-coherence.xml"
    2007-11-07 09:49:43.188 Oracle Coherence 3.3.1/389p1 <Info> (thread=Main Thread, member=n/a): Loaded operational overrides from file "/home/www/WEB-INF/lib/tangosol-coherence-override.xml"
    Oracle Coherence Version 3.3.1/389p1
    Grid Edition: Development mode
    Copyright (c) 2000-2007 Oracle. All rights reserved.
    2007-11-07 09:49:43.528 Oracle Coherence GE 3.3.1/389p1 <Info> (thread=Main Thread, member=n/a): Loaded cache configuration from file "/home/www/WEB-INF/lib/pub-search-cache-config.xml"
    2007-11-07 09:49:43.571 Oracle Coherence GE 3.3.1/389p1 <Info> (thread=Main Thread, member=n/a): sun.misc.AtomicLong is not supported on this JVM; using a synchronized counter. Though safe to ignore, you may upgrade to BEA's 1.5 JVM to fix this issue.
    2007-11-07 09:49:44.829 Oracle Coherence GE 3.3.1/389p1 <Warning> (thread=Main Thread, member=n/a): UnicastUdpSocket failed to set receive buffer size to 1428 packets (2096304 bytes); actual size is 89 packets (131071 bytes). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
    2007-11-07 09:49:45.419 Oracle Coherence GE 3.3.1/389p1 <D5> (thread=Cluster, member=n/a): Service Cluster joined the cluster with senior service member n/a
    2007-11-07 09:49:45.555 Oracle Coherence GE 3.3.1/389p1 <Info> (thread=Cluster, member=n/a): Failed to satisfy the variance: allowed=16, actual=47
    2007-11-07 09:49:45.555 Oracle Coherence GE 3.3.1/389p1 <Info> (thread=Cluster, member=n/a): Increasing allowable variance to 19
    2007-11-07 09:49:46.040 Oracle Coherence GE 3.3.1/389p1 <Info> (thread=Cluster, member=n/a): This Member(Id=2, Timestamp=2007-11-07 09:49:45.69, Address=10.5.176.85:8088, MachineId=48981, Edition=Grid Edition, Mode=Development, CpuCount=4, SocketCount=2) joined cluster with senior Member(Id=1, Timestamp=2007-11-07 09:45:10.205, Address=10.5.176.86:8088, MachineId=48982, Edition=Grid Edition, Mode=Development, CpuCount=4, SocketCount=2)
    Could you please explain why it happens, and what should I do to resolve this issues ?
    Many Thanks,
    Bing

    Hi, Gene
    Thank you for the response. I will send you our full log files and thread dumps.
    I just want to give you more details about our cases :
    1. This only happened without starting cache servers (com.tangosol.net.DefaultCacheServer).
    2. And our application which are running on weblogic cluster will just call "CacheFactory.getCache("XXX")", and running as the Tangosol DataClient.
    3. All weblogic member servers will be up running successfully if our cache servers are up running.
    Also I tried to test another case :
    Suppose all weblogic instances and cache server instances are up running. Now I trying to restart (kill weblogic instance process and restart) one of the weblogic member, It will up running successfully only if add some sleep times after killing weblogic processes and restarting it. Looks like tangosol cluster need certain time to aware the member has left cluster, then the restart process will be successful.
    Questions :
    1. Should we start our weblogic cluster only after cache server cluster is up running ?
    2. How do we decide how many time we should wait before start new process to join the cache cluster ?
    Could you please help to explain this one for me and let us if there anyway we can do to avoid the problem.
    Many Thanks !!!
    Bing

  • Starting weblogic cluster and admin server on Win 2000 platform

    Hi! Here's my problem scenario:
    I'm running my WLS 8.1 application as WLS cluster on Win 2000 platform. I have
    3 replicated WLS servers in my cluster and of course one WLS server as admin server
    (4 WLS servers all together). I'm starting my WLS application using Win services
    which behind a scene call win 2000 comand interpreter batch-file to start the
    servers. Currently I have 4 different win 2000 services each calling different
    batch-files to start a WLS server (4 all together). Here's my question:
    Is there any way to start my cluster (3 managed WLS servers) and WLS admin server
    from single win 2000 command file (using single win 2000 service). Using win 2000
    AT-command interpreter command I can start executables and command(s) and command-files
    at background at specific date and time, but I want to start my WLS application
    on every win 2000 platform boot, is there any way to achieve this ???
    Best Regards!
    Jami

    hi,
    Thanks for stepping up!
    But I wasn't using any clustering or loadbalancing.
    Just a simple Admin server, Managed server set up.
    256 Meg, Pentium box running Win 2000 premium edition.
    I tried tweeking around the heap size too.... coz' the operations were slogging
    like anything...
    Bal
    "Raja Mukherjee" <[email protected]> wrote:
    It seems like network issue....do you have Windows Clustering/Load Balancing
    service turned on by any chance? I would also use netmon to see the traffic
    between the ips....
    ..raja
    "Bala" <[email protected]> wrote in message
    news:3b3a6c00$[email protected]..
    hi,
    I am looking at a strange behaviour with WLS6.0SP1 installed on WINDOWS2000 Premium.
    I brought up the Admin server and a Managed server. After a while theAdmin Server
    is compeletely
    losing the control of the managed server (for no reason).
    when I tried monitoring the managed server from the console, I couldnot
    access
    the manager server
    and the status down (red ball). I tried restarting the Admin Server
    in the discover mode - NO LUCK.
    But when I restart the Managed server and go to the Web Based Admin
    Console and refresh, it worked. But I dont think we should be doingthat
    in a
    production environment.
    Did anybody come across such a problem before ???
    Is this a problem with Win 2000 ???
    coz' it never happened with NT workstation
    thanks in advance

  • Programmatically start/stop applications without admin server connection

    I know I can programmatically start/stop an application through WebLogic's deployment API, but the call needs to go through admin server. I was wondering if there's a way for an application to start/stop another application on the same managed server when the admin server is down. I remember a managed server could be started without an admin server (unless it's started for the first time). So I suppose there must be some way to programmatically start/stop local applications without an admin server as well.
    Thank you for your help in advance.

    Hai,
    I read the Oracle weblogic wlst script document without Admin server you cannot connect the managed server through the WLST script. see the below lines
    "The start command starts Managed Servers or clusters in a domain using Node Manager.
    To use the start command, WLST must be connected to a running Administration Server.
    To start Managed Servers without requiring a running Administration Server, use the
    nmStart command with WLST connected to Node Manager."
    "You shut down a server to which WLST is connected by entering the shutdown command
    without any arguments.
    When connected to a Managed Server instance, you only use the shutdown command to shut
    down the Managed Server instance to which WLST is connected; you cannot shut down another
    server while connected to a Managed Server instance.
    WLST uses Node Manager to shut down a Managed Server. When shutting down a Managed
    Server, Node Manager must be running.
    In the event of an error, the command returns"
    They are two option if you are using adminserver then we can stop the any Managed server.
    The option is if you are using the nodemanager without admin server we can stop the any Managed server.
    The last final solution to kill the particular Managed server pid.
    Regards,
    S.vinoth babu

  • Is it possible to programmatically start the CAN Bus Monitor?

    Is it possible to programmatically start the CAN Bus Monitor?  I was unable to find the executable, so either I'm not looking in the right place or it's built into NI-CAN or MAX.
    Message Edited by FlamingYawn on 05-24-2007 06:19 PM
    Robert C. Mortensen
    Certified LabVIEW Architect
    Certified LabVIEW Embedded Systems Developer
    Endigit

    Indeed it is.  I found the executable at c:\Program Files\National Instruments\NI-CAN\bin.  It won't run by simply clicking on it, you must call it with the can port (e.g. CAN0) as the first paramenter.
    Robert C. Mortensen
    Certified LabVIEW Architect
    Certified LabVIEW Embedded Systems Developer
    Endigit

  • Start a cluster locally with 2 nodes

    Hi,
    I just want to ask that how to start a cluster locally with 2 nodes?
    By default i think that it starts only 1.
    Thanks

    Hi,
    Just start another instance of the same process using the same coherence configuration and they will form a cluster.
    mark

  • Windows could not start the Cluster Service on Local computer. For more information, review the System Event Log. If this is a non-Microsoft service, contact the service vendor, and refer to service-specific error code 2.

    Dear Technet,
    Windows could not start the Cluster Service on Local computer. For more information, review the System Event Log. If this is a non-Microsoft service, contact the service vendor, and refer to service-specific error code 2.
    My cluster suddenly went disappear. and tried to restart the cluster service. When trying to restart service this above mention error comes up.
    even i tried to remove the cluster through power-shell still couldn't happen because of cluster service not running.
    Help me please.. thank you.
    Regards
    Shamil

    Hi,
    Could you confirm which account when you start the cluster service? The Cluster service is a service that requires a domain user account.
    The server cluster Setup program changes the local security policy for this account by granting a set of user rights to the account. Additionally, this account is made a member
    of the local Administrators group.
    If one or more of these user rights are missing, the Cluster service may stop immediately during startup or later, depending on when the Cluster service requires the particular
    user right.
    Hope this helps.
    We
    are trying to better understand customer views on social support experience, so your participation in this
    interview project would be greatly appreciated if you have time.
    Thanks for helping make community forums a great place.

  • Scripting the start of cluster-caching

    Hello,
    im a SAP Portal administrator. In the Portal runs a Jive Froumssoftware with tangosol coherence caching feature. In the forumssoftware I can activate the clsuter caching, but after a serverrestart i have to activate it again.
    I know this is not an SAP Forum oder Jive Forum but I want to know how i can script the activation of the cluster?
    I think there must be a way to make ist via command line. In wich class or jar file i can have a look to start the cluster-caching by scripting? E.g: coherence.jar?
    Is there a way like: java -cp tangosol.jar com.tangosol.net.MulticastTest (run Multicasttest) to activate multicast?
    thx and sry if i post wrong

    I am not knowledgeable about the Jive Forums cluster administration. (I'm pretty sure that these forums run on Jive, but they're managed by our IT organization.)
    I would suggest posting the question on the Jive support forums:
    http://www.jivesoftware.com/jivespace/community/support/jive_forums?view=discussions
    Peace,
    Cameron Purdy | Oracle Coherence

  • Can not start the cluster database using srvctl

    Created a RAC database with CRS.
    When i start the cluster database with srvctl, it appears that the database is started (i made sure using srvctl status), but when i try to login to the database using sqlplus, i get a message 'connected to idle instance'. if i try to start the database from sqlplus, i get message 'requested INSTANCE_NUMBER is busy!.
    any thoughts why i can not connect to the database using sqlplus after i start the database using srvctl?
    I stop the database using srvctl and started using traditional sqlplus, it will start with out any problem.
    what am i missing?

    check ur oracle_sid,also try to connect as sqlplus sys/****@prod as sysdba
    may be this is due to improper setting of init.ora file.
    and post ur sqlplus output here.
    Thanks
    Kuljeet Pal Singh

  • DB2 is not starting the Cluster environment

    Hi, Experts
    we are using HP Unix Service Guard cluster envirnment for our
    production server.
    I have sucessfully installed SAP SR3 in this environment. When the
    failover happen, then filesystem is automatically moving. when i try to
    start in Node B.
    It's throwing error "
    02/12/2009 16:09:31 0 0 SQL6048N A communication error
    occurred during
    START or STOP DATABASE MANAGER processing.
    02/12/2009 16:09:32 1 0 SQL6048N A communication error
    occurred during
    START or STOP DATABASE MANAGER processing.
    SQL1032N No start database manager command was issued. SQLSTATE=57019"But it's working fine in Node A. I am attaching the db2diag.log and
    trans.log for your reference.
    In db2diag.log
    2009-02-02-10.42.05.176811+330 I1A1279            LEVEL: Event
    PID     : 2928                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, RAS/PD component, pdLogInternal, probe:120
    START   : New Diagnostic Log file
    DATA #1 : Build Level, 152 bytes
    Instance "db2sop" uses "64" bits and DB2 code release "SQL09016"
    with level identifier "01070107".
    Informational tokens are "DB2 v9.1.0.6", "s081007", "U817477", Fix Pack "6".
    DATA #2 : System Info, 1568 bytes
    System: HP-UX SOGLPRDP B.11.31 U ia64
    CPU: total:8 online:4 Threading degree per core:1
    Physical Memory(MB): total:16363 free:11923
    Virtual  Memory(MB): total:65515 free:61075
    Swap     Memory(MB): total:49152 free:49152
    Kernel   Params: msgMaxMessageSize:65535 msgMsgMap:4098 msgMaxQueueIDs:4096
                     msgNumberOfHeaders:4096 msgMaxQueueSize:65535
                     msgMaxSegmentSize:248 shmMax:17179869184 shmMin:1 shmIDs:512
                     shmSegments:300 semMap:8194 semIDs:8192 semNum:16384
                     semUndo:4092 semNumPerID:2048 semOps:500 semUndoPerProcess:100
                     semUndoSize:824 semMaxVal:32767 semAdjustOnExit:16384
    Information in this record is only valid at the time when this file was
    created (see this record's time stamp)
    2009-02-02-10.42.38.940922+330 I4655A1629         LEVEL: Event
    PID     : 3883                 TID  : 1           PROC : db2stop
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, base sys utilities, sqleStartStopSingleNode, probe:1130
    DATA #1 : String, 31 bytes
    /db2/db2sop/sqllib/adm/db2stop2
    DATA #2 : Hexdump, 256 bytes
    0x9FFFFFFFFFFFAD60 : 2F64 6232 2F64 6232 736F 702F 7371 6C6C    /db2/db2sop/sqll
    0x9FFFFFFFFFFFAD70 : 6962 2F61 646D 2F64 6232 7374 6F70 3200    ib/adm/db2stop2.
    0x9FFFFFFFFFFFAD80 : 4E4F 4D53 4700 464F 5243 4500 534E 0000    NOMSG.FORCE.SN..
    0x9FFFFFFFFFFFAD90 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFADA0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFADB0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFADC0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFADD0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFADE0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFADF0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFAE00 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFAE10 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFAE20 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFAE30 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFAE40 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x9FFFFFFFFFFFAE50 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    2009-02-02-10.42.49.871047+330 I6285A289          LEVEL: Event
    PID     : 4005                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Dir_cache" From: "1"  To: "0"
    2009-02-02-10.42.49.934583+330 I6575A306          LEVEL: Event
    PID     : 4006                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Sysadm_group" From: "DBSOPADM"  To: "dbsopadm"
    2009-02-02-10.42.50.000401+330 I6882A299          LEVEL: Event
    PID     : 4007                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Sysctrl_group" From: ""  To: "dbsopctl"
    2009-02-02-10.42.50.062186+330 I7182A300          LEVEL: Event
    PID     : 4008                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Sysmaint_group" From: ""  To: "dbsopmnt"
    2009-02-02-10.42.50.127927+330 I7483A295          LEVEL: Event
    PID     : 4009                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Dft_mon_bufpool" From: "0"  To: "1"
    2009-02-02-10.42.50.203836+330 I7779A292          LEVEL: Event
    PID     : 4012                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Dft_mon_lock" From: "0"  To: "1"
    2009-02-02-10.42.50.265351+330 I8072A292          LEVEL: Event
    PID     : 4021                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Dft_mon_sort" From: "0"  To: "1"
    2009-02-02-10.42.50.329484+330 I8365A292          LEVEL: Event
    PID     : 4022                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Dft_mon_stmt" From: "0"  To: "1"
    2009-02-02-10.42.50.390840+330 I8658A293          LEVEL: Event
    PID     : 4023                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Dft_mon_table" From: "0"  To: "1"
    2009-02-02-10.42.50.514969+330 I8952A291          LEVEL: Event
    PID     : 4025                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Dft_mon_uow" From: "0"  To: "1"
    2009-02-02-10.42.50.576102+330 I9244A294          LEVEL: Event
    PID     : 4026                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Mon_heap_sz" From: "90"  To: "128"
    2009-02-02-10.42.50.701560+330 I9539A296          LEVEL: Event
    PID     : 4028                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Rqrioblk" From: "32767"  To: "65000"
    2009-02-02-10.42.50.761306+330 I9836A295          LEVEL: Event
    PID     : 4029                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Svcename" From: ""  To: "sapdb2SOP"
    2009-02-02-10.42.50.826115+330 I10132A291         LEVEL: Event
    PID     : 4030                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Aslheapsz" From: "15"  To: "16"
    2009-02-02-10.42.50.886807+330 I10424A290         LEVEL: Event
    PID     : 4031                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Keepfenced" From: "1"  To: "0"
    2009-02-02-10.42.50.951758+330 I10715A294         LEVEL: Event
    PID     : 4032                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Maxagents" From: "400"  To: "1024"
    2009-02-02-10.42.51.017562+330 I11010A295         LEVEL: Event
    PID     : 4033                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Dftdbpath" From: ""  To: "/db2/SOP"
    2009-02-02-10.42.51.078023+330 I11306A328         LEVEL: Event
    PID     : 4034                 TID  : 1           PROC : db2flacc
    INSTANCE: db2sop               NODE : 000
    FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:30
    CHANGE  : CFG DBM: "Diagpath" From: "/db2/db2sop/sqllib/db2dump"  To: "/db2/SOP/db2dump"
    2009-02-10-15.23.47.236724+330 I11635A213         LEVEL: Error
    PID     : 25606                TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-10-16.28.21.589909+330 I11849A213         LEVEL: Error
    PID     : 6745                 TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-10-18.38.17.375623+330 I12063A213         LEVEL: Error
    PID     : 16296                TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-10-19.53.31.036205+330 I12277A213         LEVEL: Error
    PID     : 6044                 TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-11-23.37.20.156996+330 I12491A213         LEVEL: Error
    PID     : 15427                TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-11-23.57.56.959811+330 I12705A213         LEVEL: Error
    PID     : 4895                 TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-12-00.42.20.199428+330 I12919A213         LEVEL: Error
    PID     : 5424                 TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-12-00.55.15.231691+330 I13133A213         LEVEL: Error
    PID     : 4065                 TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-12-01.23.40.283293+330 I13347A213         LEVEL: Error
    PID     : 4345                 TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-12-01.59.06.153079+330 I13561A213         LEVEL: Error
    PID     : 4145                 TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-12-02.40.06.335045+330 I13775A213         LEVEL: Error
    PID     : 4515                 TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-12-03.09.22.341443+330 I13989A213         LEVEL: Error
    PID     : 4423                 TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    2009-02-12-03.31.31.201911+330 I14203A213         LEVEL: Error
    PID     : 4135                 TID : 1
    FUNCTION: DB2 Common, Generic Control Facility, gcf_stop, probe:60
    MESSAGE : ECF=0x90000390 Invalid process id
    In R3trans.log
    4 ETW000  [dev trc     ,00000]    (0.144) MSSQL: ODBC fastload on separate connection (note 1131805)
    4 ETW000                                                                              46  0.514332
    4 ETW000  [dev trc     ,00000]  Supported features:                                   30  0.514362
    4 ETW000  [dev trc     ,00000]  ..retrieving configuration parameters                 29  0.514391
    4 ETW000  [dev trc     ,00000]  ..done                                              3613  0.518004
    4 ETW000  [dev trc     ,00000]  Running with UTF-8 Unicode                            30  0.518034
    4 ETW000  [dev trc     ,00000]  Running with CLI driver                           126609  0.644643
    4 ETW000  [dev trc     ,00000]  *** ERROR in DB6Connect[dbdb6.c, 1632] CON = 0 (BEGIN)
    4 ETW000                                                                          180751  0.825394
    4 ETW000  [dev trc     ,00000]  &+     DbSlConnectDB6( SQLConnect ): [IBM][CLI Driver] SQL30081N  A communication error has been detected.  Communication
    4 ETW000                                                                              57  0.825451
    4 ETW000  [dev trc     ,00000]  &+      protocol being used: "TCP/IP".  Communication API being used: "SOCKETS".  Location where the error was detected:
    4 ETW000                                                                              49  0.825500
    4 ETW000  [dev trc     ,00000]  &+     "128.0.0.35".  Communication function detecting the error: "connect".  Protocol specific error code(s): "239", "*"
    &+                                                                               
    4 ETW000                                                                              49  0.825978
    4 ETW000  [dev trc     ,00000]  *** ERROR in DB6Connect[dbdb6.c, 1632] (END)          30  0.826008
    4 ETW000  [dbdb6.c     ,00000]  *** ERROR => DbSlConnect to 'SOP' as 'sapsop' failed
    4 ETW000                                                                             554  0.826562
    2EETW169 no connect possible: "DBMS = DB6                              --- DB2DBDFT = 'SOP'"
    Please provide me the solution to solve the issue.
    Thanks & Regards

    Hi, Malte
    I have given individual nodes address in db2nodes.cfg,  as per your instruction I have run the db2 list database directory I have got the following output
    System Database Directory
    Number of entries in the directory = 1
    atabase 1 entry:
    Database alias                       = SOP
    Database name                        = SOP
    Local database directory             = /db2/SOP
    Database release level               = b.00
    Comment                              = SAP database SOP
    Directory entry type                 = Indirect
    Catalog database partition number    = 0
    Alternate server hostname            =
    Alternate server port number         =
    => db2 list node directory
    => SQL1027N  The node directory cannot be found
    Regards

  • BAM server does not start after cluster install

    Hi All,
    BAM server is not starting properly with the below exception. It happens in cluster environment linux 5.5, single node linux 5.5 enviroment. I am using jrockit JDK.
    Its Urgent. Please let me know if some one has come across this issue.
    <Feb 21, 2012 10:43:54 PM EST> <Error> <oracle.bam.adc.kernel.viewsets.satellite.GenericSatellite> <BEA-000000> <BAM-00553: Resources start operation for destination topic/oracle.bam.messaging.systemobjectnotification failed.. [Ljava.lang.StackTraceElement;@2ab016f>
    <Feb 21, 2012 10:43:54 PM EST> <Error> <HTTP> <BEA-101216> <Servlet: "BamServer" failed to preload on startup in Web application: "/oracle/bam".
    java.lang.ExceptionInInitializerError
         at oracle.bam.adc.kernel.viewsets.satellite.GenericSatellite.start(GenericSatellite.java:285)
         at oracle.bam.adc.kernel.server.ADCService.startService(ADCService.java:88)
         at oracle.bam.server.BamServer$ServiceEntry$1.run(BamServer.java:1717)
         at javax.security.auth.Subject.doAsPrivileged(Subject.java:517)
         at oracle.security.jps.internal.jaas.AccActionExecutor.execute(AccActionExecutor.java:47)
         Truncated. see log file for complete stacktrace
    Caused By: oracle.bam.common.messaging.api.exception.StartFailedException: BAM-00553: Resources start operation for destination topic/oracle.bam.messaging.systemobjectnotification failed.
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.onStartFailedException(MessageOperationBase.java:1616)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.start(MessageOperationBase.java:1521)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.start(MessageOperationBase.java:1476)
         at oracle.bam.adc.kernel.viewsets.satellite.GenericSatellite.start(GenericSatellite.java:274)
         at oracle.bam.adc.kernel.server.ADCService.startService(ADCService.java:85)
         Truncated. see log file for complete stacktrace
    Caused By: oracle.bam.common.messaging.api.MessageFrameworkException: BAM-00500: exception in messaging framework: BAM-00533: Resource open for destination topic/oracle.bam.messaging.systemobjectnotification failed.
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.onStartException(MessageOperationBase.java:1600)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.performStart(MessageOperationBase.java:1572)
         at oracle.bam.common.messaging.api.jms.JMSMessageProducer.performStart(JMSMessageProducer.java:513)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.start(MessageOperationBase.java:1512)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.start(MessageOperationBase.java:1477)
         Truncated. see log file for complete stacktrace
    Caused By: oracle.bam.common.messaging.api.exception.OpenFailedException: BAM-00533: Resource open for destination topic/oracle.bam.messaging.systemobjectnotification failed.
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.waitForOpen(MessageOperationBase.java:1355)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.waitForOpen(MessageOperationBase.java:1290)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.performStart(MessageOperationBase.java:1560)
         at oracle.bam.common.messaging.api.jms.JMSMessageProducer.performStart(JMSMessageProducer.java:513)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.start(MessageOperationBase.java:1512)
         Truncated. see log file for complete stacktrace
    Caused By: oracle.bam.common.messaging.api.exception.OpenFailedException: BAM-00533: Resource open for destination topic/oracle.bam.messaging.systemobjectnotification failed.
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.onOpenFailedException(MessageOperationBase.java:1272)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.open(MessageOperationBase.java:1172)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.access$000(MessageOperationBase.java:72)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase$1.performOperation(MessageOperationBase.java:1152)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase$UtilityThread.run(MessageOperationBase.java:2545)
    Caused By: oracle.bam.common.messaging.api.exception.RecoveryFailedException: BAM-00523: Resource recovery for destination topic/oracle.bam.messaging.systemobjectnotification failed.
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.onRecoveryFailedException(MessageOperationBase.java:2027)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.recover(MessageOperationBase.java:1865)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.onOpenException(MessageOperationBase.java:1251)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.performOpen(MessageOperationBase.java:1222)
         at oracle.bam.common.messaging.api.invmjms.INVMJMSMessageProducer.performOpen(INVMJMSMessageProducer.java:222)
         Truncated. see log file for complete stacktrace
    Caused By: oracle.bam.common.messaging.api.exception.RecoveryFailedException: BAM-00523: Resource recovery for destination topic/oracle.bam.messaging.systemobjectnotification failed.
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.performRecovery(MessageOperationBase.java:1979)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.recover(MessageOperationBase.java:1856)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.onOpenException(MessageOperationBase.java:1252)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.performOpen(MessageOperationBase.java:1222)
         at oracle.bam.common.messaging.api.invmjms.INVMJMSMessageProducer.performOpen(INVMJMSMessageProducer.java:223)
         Truncated. see log file for complete stacktrace
    Caused By: oracle.bam.common.messaging.api.MessageFrameworkException: BAM-00500: exception in messaging framework: BAM-00500: exception in messaging framework: null
         at oracle.bam.common.messaging.api.jms.JMSMessageProducer.acquireResources(JMSMessageProducer.java:698)
         at oracle.bam.common.messaging.api.invmjms.INVMJMSMessageProducer.acquireResources(INVMJMSMessageProducer.java:282)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.performOpen(MessageOperationBase.java:1211)
         at oracle.bam.common.messaging.api.invmjms.INVMJMSMessageProducer.performOpen(INVMJMSMessageProducer.java:223)
         at oracle.bam.common.messaging.base.operation.MessageOperationBase.open(MessageOperationBase.java:1163)
         Truncated. see log file for complete stacktrace
    Caused By: oracle.bam.common.messaging.api.MessageFrameworkException: BAM-00500: exception in messaging framework: null
         at oracle.bam.common.messaging.base.connection.jms.JMSConnectionFactory.newInstance(JMSConnectionFactory.java:336)
         at oracle.bam.common.messaging.base.connection.jms.JMSSharedConnectionFactory.newInstance(JMSSharedConnectionFactory.java:279)
         at oracle.bam.common.messaging.base.connection.jms.JMSSharedConnectionFactory.newInstance(JMSSharedConnectionFactory.java:55)
         at oracle.bam.common.messaging.base.connection.ConnectionFactoryBase.getInstance(ConnectionFactoryBase.java:84)
         at oracle.bam.common.messaging.base.connection.jms.JMSSharedConnectionFactory.getInstance(JMSSharedConnectionFactory.java:149)
         Truncated. see log file for complete stacktrace
    Caused By: javax.naming.CommunicationException [Root exception is java.net.ConnectException: t3://CHAITU-HP:9001: Destination unreachable; nested exception is:
         java.net.ConnectException: Connection refused: connect; No available router to destination]
         at weblogic.jndi.internal.ExceptionTranslator.toNamingException(ExceptionTranslator.java:40)
         at weblogic.jndi.WLInitialContextFactoryDelegate.toNamingException(WLInitialContextFactoryDelegate.java:788)
         at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:368)
         at weblogic.jndi.Environment.getContext(Environment.java:315)
         at weblogic.jndi.Environment.getContext(Environment.java:285)
         Truncated. see log file for complete stacktrace
    Caused By: java.net.ConnectException: t3://CHAITU-HP:9001: Destination unreachable; nested exception is:
         java.net.ConnectException: Connection refused: connect; No available router to destination
         at weblogic.rjvm.RJVMFinder.findOrCreateInternal(RJVMFinder.java:216)
         at weblogic.rjvm.RJVMFinder.findOrCreate(RJVMFinder.java:170)
         at weblogic.rjvm.ServerURL.findOrCreateRJVM(ServerURL.java:153)
         at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:353)
         at weblogic.jndi.Environment.getContext(Environment.java:315)
         Truncated. see log file for complete stacktrace
    Caused By: java.rmi.ConnectException: Destination unreachable; nested exception is:
         java.net.ConnectException: Connection refused: connect; No available router to destination
         at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:470)
         at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:321)
         at weblogic.rjvm.RJVMManager.findOrCreateRemoteInternal(RJVMManager.java:254)
         at weblogic.rjvm.RJVMManager.findOrCreate(RJVMManager.java:197)
         at weblogic.rjvm.RJVMFinder.findOrCreateRemoteServer(RJVMFinder.java:238)
         Truncated. see log file for complete stacktrace
    -Chaitu

    The issue was with JMS modules. I missed to identify the subdeployments tab for UMSJMSSystemResource where we had two sub deployments as UMSJMSSubDMSOA and UMSJMSSubDMSOA_auto..
    I've removed the second subdeployment and all resources listed against is by following this below path in Admin Console
    click JMS Modules -> Click UMSJMSSystemResource -> Click the Subdeployments tab.
    Ensure that UMSJMSSubDMSOA is the only subdeployment available. Remove others if listed and all dependant resources as applicable to your environment.
    Go back to UMSJMSSystemResource and click Targets.
    Deselect BAM_Cluster as target.
    Click Save and Activate.
    Upon subsequent restart, the Server came up nice and fine.
    Regards!
    Sarat

  • DS 6.1 won't start as cluster service

    Hi!
    RG: hylas-ldap
    Resources:
    Logical Hostname hylas
    HAStorage ldap-data-1 (Diskgroup contains instance data)
    I installed DS 6.0 and patched up to 6.1. As the topic says... I'm able to start the instance on every node of the Cluster (SC 3.2) with "dsadm start /path/to/instance". Prior to that I switched the RG over to the node where the DS is started manually.
    Once I register the SUNW.ds6ldap instance in my RG, it won't start.
    Log:
    Jul 31 11:57:55 xxxx SC[SUNW.ds6ldap,hylas-ldap,ds--argolis-disks-ldapdata1-hylas,dsldap_svc_start]: [ID 833212 daemon.info] Attempting to start the data service under process monitor facility.
    Jul 31 11:57:55 xxxx Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource ds--argolis-disks-ldapdata1-hylas status on node xxxx change to R_FM_ONLINE
    Jul 31 11:57:55 xxxx Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource ds--argolis-disks-ldapdata1-hylas status msg on node xxxx change to <Completed successfully.>
    Jul 31 11:57:55 xxxx Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="hylas-ldap,ds--argolis-disks-ldapdata1-hylas,0.svc", cmd="/bin/sh -c /opt/SUNWdsee/ds6/bin/dsadm start /argolis/disks/ldapdata1/hylas", Failed to stay up.
    I tried to execute the bold commandline... it quits with "Error: Missing subcommand".
    Adding the missing quotes "/bin/sh -c \"/opt/SUNWdsee/ds6/bin/dsadm start /argolis/disks/ldapdata1/hylas\" would certainly help, but where do I apply the fix?
    Maybe thats not the problem... any hint is appreciated.
    TIA && rgds,
    orknase

    Hi, ludovicp!
    dsadm start exec /path/to/instance works fine. The only time it doesn't start up is when I enable dsargolis-disks-ldapdata1-hylas in RG hylas-ldap and switch the RG online.
    snip taken after dsadm start ...
    dsadm info /argolis/disks/ldapdata1/hylas
    Instance Path:               /argolis/disks/ldapdata1/hylas
    Owner:                       root(root)
    Non-secure port:             389
    Secure port:                 636
    Bit format:                  64-bit
    State:                       Could be retrieved using Sun Cluster commands.
    Server PID:                  12191
    DSCC url:                    -
    Sun Cluster resource group:  hylas-ldap
    SMF application name:        -
    Instance version:            D-A00So yes, it looks like a problem with the interaction SC/DS. The agent submits the same command. Thats why I thought the defect #6563445 is related to it. Oh well, I think on monday the service plan information will be available to this account and I will have a few more possibilities to check.
    Rgds,
    orknase
    Message was edited by:
    orknase

  • Getting ssh error while starting 10g cluster installation

    Hi frnds
    I am getting the following error while running 10g clusterware installation.Actually i done the ssh setup mentioned in the oracle doc.Generated both keys (rsa,dsa) on both nodes and copied these 2 files from both nodes to authorized_key file and copy this file to both nodes and ran $shell and ssh-add.Whenever i logs and run this SSH User Equivalency on node1(from where i run oui) and its persisting with other sessions but if logoff and logon i have to run the $shell and ssh-add to enable it.My dought is whether this ssh have to enable always or only while using OUI to install and applying patches cos this Equivalency dont need to run on second node correct ?
    Caught Cluster Exception PRKC-1044 : Failed to check remote command execution setup for node racdb1.bsa.net.in using shells /usr/bin/ssh and /usr/bin/rsh
    racdb1.bsa.net.in: Connection refused
    rgds
    rajesh

    Hi
    thnks for that but when i run that exec $SHELL and ssh-add again in one session i can do ssh it other sessions without any problem but if i logoff and login it will reset to back.Whether this ssh have to enabled on first node(where i run OUI) always irrespective of logons and logout or whether i have to start any services ?
    rgds
    rajesh

  • StreamCorruptedException when starting Weblogic cluster

    Hi, I have an application running on Weblogic Server 8.1 sp4 in clustered environment. When the cluster is restarted the first server restarts without errors but the second one has the following error message. What does the error message mean? Some messages have disappeared form the JMS queue when the cluster has been restarted. Does this error message have something to do with the disappeared messages?
              <29.9.2006 14:09:17 EEST> <Notice> <Cluster> <BEA-000138> <Listening for announcements from cluster Cluster2 on 237.0.0.6:10603.>
              <29.9.2006 14:09:17 EEST> <Notice> <Cluster> <BEA-000133> <Waiting to synchronize with other running members of Cluster2.>
              <29.9.2006 14:09:47 EEST> <Notice> <Cluster> <BEA-000142> <Trying to download cluster JNDI tree from server CL2Server2.>
              <29.9.2006 14:09:54 EEST> <Error> <JMS> <BEA-040368> <The following exception has occurred:
              java.io.StreamCorruptedException: Unknown object stream version. 5505025
                   at weblogic.jms.store.BufferDataInputStream.readObject(BufferDataInputStream.java:167)
                   at weblogic.jms.store.JDBCIOStream.doRecoverBodies(JDBCIOStream.java:1024)
                   at weblogic.jms.store.JDBCIOStream.doRecover(JDBCIOStream.java:939)
                   at weblogic.jms.store.JDBCIOStream.recover(JDBCIOStream.java:1140)
                   at weblogic.jms.store.JMSStore.recover(JMSStore.java:315)
                   at weblogic.jms.backend.BEStore.open(BEStore.java:264)
                   at weblogic.jms.backend.BEStore.start(BEStore.java:151)
                   at weblogic.jms.backend.BackEnd.openStores(BackEnd.java:1171)
                   at weblogic.jms.backend.BackEnd.resume(BackEnd.java:1290)
                   at weblogic.jms.backend.BackEnd.migratableActivate(BackEnd.java:2939)
                   at weblogic.cluster.migration.MigratableGroup.add(MigratableGroup.java:107)
                   at weblogic.cluster.migration.MigrationManager.privateRegister(MigrationManager.java:180)
                   at weblogic.cluster.migration.MigrationManager.register(MigrationManager.java:127)
                   at weblogic.jms.JMSService.addJMSServer(JMSService.java:2226)
                   at weblogic.jms.JMSService.addDeployment(JMSService.java:2031)
                   at weblogic.management.mbeans.custom.DeploymentTarget.addDeployment(DeploymentTarget.java:337)
                   at weblogic.management.mbeans.custom.DeploymentTarget.addDeployments(DeploymentTarget.java:597)
                   at weblogic.management.mbeans.custom.DeploymentTarget.updateServerDeployments(DeploymentTarget.java:575)
                   at weblogic.management.mbeans.custom.DeploymentTarget.updateDeployments(DeploymentTarget.java:241)
                   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                   at java.lang.reflect.Method.invoke(Method.java:324)
                   at weblogic.management.internal.DynamicMBeanImpl.invokeLocally(DynamicMBeanImpl.java:754)
                   at weblogic.management.internal.DynamicMBeanImpl.invoke(DynamicMBeanImpl.java:733)
                   at weblogic.management.internal.ConfigurationMBeanImpl.invoke(ConfigurationMBeanImpl.java:509)
                   at com.sun.management.jmx.MBeanServerImpl.invoke(MBeanServerImpl.java:1560)
                   at com.sun.management.jmx.MBeanServerImpl.invoke(MBeanServerImpl.java:1528)
                   at weblogic.management.internal.RemoteMBeanServerImpl.private_invoke(RemoteMBeanServerImpl.java:988)
                   at weblogic.management.internal.RemoteMBeanServerImpl.invoke(RemoteMBeanServerImpl.java:946)
                   at weblogic.management.internal.MBeanProxy.invoke(MBeanProxy.java:954)
                   at weblogic.management.internal.MBeanProxy.invokeForCachingStub(MBeanProxy.java:481)
                   at weblogic.management.configuration.ServerMBean_Stub.updateDeployments(ServerMBean_Stub.java:7691)
                   at weblogic.management.deploy.slave.SlaveDeployer.updateServerDeployments(SlaveDeployer.java:1304)
                   at weblogic.management.deploy.slave.SlaveDeployer.resume(SlaveDeployer.java:347)
                   at weblogic.management.deploy.DeploymentManagerServerLifeCycleImpl.resume(DeploymentManagerServerLifeCycleImpl.java:229)
                   at weblogic.t3.srvr.SubsystemManager.resume(SubsystemManager.java:131)
                   at weblogic.t3.srvr.T3Srvr.resume(T3Srvr.java:966)
                   at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:361)
                   at weblogic.Server.main(Server.java:32)

    Make sure that the JMS server isn't attempting to load a store that was created from a later version of WLS. Also, make sure that no two JMS servers share the same database table - otherwise they will corrupt each-other's data. I think the latter is likely the problem.
              Tom

  • Managed SOA Server not started in cluster environment.

    Hi,
    I am trying to make a SOA cluster environment on Linux based machines. After doing domain configuration which also involved packing of domain on machine 1 and unpacking on machine2, I tried to start the servers. I'm able to start AdminServer successfully and also the managed soa server on the same machine. When I'm trying to start managed soa server on other machine, I'm getting the following error..
    *<Unable to establish JMX Connectivity with the Adminstration Server AdminServer at service:jmx:t3://192.168.122.1:7001/jndi/weblogic.management.mbeanservers.domainruntime.*
    java.io.IOException>
    My machine IP is XXX.XX.XXX.41 for Admin and XXX.XX.XXX.42 for Machine2. I guess the IP 192.168.122.1 is being used for cluster connection by Machine2 to make connection to Machine1. the managed soa server is getting started after the chain of error logs but it is not starting as a part of the entire cluster and comes as shutdown in WLS Console.
    I'm using Linux 5 Update 7 OS and SOA 11.1.1.5.0. Any help will be appreciated for resolving this issue.
    Please find the logs below.
    *<Unable to establish JMX Connectivity with the Adminstration Server AdminServer at service:jmx:t3://192.168.122.1:7001/jndi/weblogic.management.mbeanservers.domainruntime.*
    java.io.IOException
    *     at weblogic.management.remote.common.ClientProviderBase.makeConnection(ClientProviderBase.java:196)*
    *     at weblogic.management.remote.common.ClientProviderBase.newJMXConnector(ClientProviderBase.java:84)*
    *     at javax.management.remote.JMXConnectorFactory.newJMXConnector(JMXConnectorFactory.java:338)*
    *     at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:247)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.getDomainMBeanServerConnection(RegisterWithDomainRuntimeService.java:222)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.notifyDomainRuntime(RegisterWithDomainRuntimeService.java:137)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.start(RegisterWithDomainRuntimeService.java:91)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeServiceLate.start(RegisterWithDomainRuntimeServiceLate.java:17)*
    *     at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)*
    *     at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)*
    *     at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)*
    *     at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)*
    *Caused By: javax.naming.CommunicationException [Root exception is java.net.ConnectException: t3://192.168.122.1:7001: Destination unreachable; nested exception is:*
    *     java.net.ConnectException: Connection refused; No available router to destination]*
    *     at weblogic.jndi.internal.ExceptionTranslator.toNamingException(ExceptionTranslator.java:40)*
    *     at weblogic.jndi.WLInitialContextFactoryDelegate.toNamingException(WLInitialContextFactoryDelegate.java:788)*
    *     at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:368)*
    *     at weblogic.jndi.Environment.getContext(Environment.java:315)*
    *     at weblogic.jndi.Environment.getContext(Environment.java:285)*
    *     at weblogic.jndi.WLInitialContextFactory.getInitialContext(WLInitialContextFactory.java:117)*
    *     at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)*
    *     at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288)*
    *     at javax.naming.InitialContext.init(InitialContext.java:223)*
    *     at javax.naming.InitialContext.<init>(InitialContext.java:197)*
    *     at weblogic.management.remote.common.ClientProviderBase.makeConnection(ClientProviderBase.java:178)*
    *     at weblogic.management.remote.common.ClientProviderBase.newJMXConnector(ClientProviderBase.java:84)*
    *     at javax.management.remote.JMXConnectorFactory.newJMXConnector(JMXConnectorFactory.java:338)*
    *     at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:247)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.getDomainMBeanServerConnection(RegisterWithDomainRuntimeService.java:222)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.notifyDomainRuntime(RegisterWithDomainRuntimeService.java:137)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.start(RegisterWithDomainRuntimeService.java:91)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeServiceLate.start(RegisterWithDomainRuntimeServiceLate.java:17)*
    *     at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)*
    *     at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)*
    *     at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)*
    *     at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)*
    Caused By: java.net.ConnectException: t3://192.168.122.1:7001: Destination unreachable; nested exception is:
    *     java.net.ConnectException: Connection refused; No available router to destination*
    *     at weblogic.rjvm.RJVMFinder.findOrCreateInternal(RJVMFinder.java:216)*
    *     at weblogic.rjvm.RJVMFinder.findOrCreate(RJVMFinder.java:170)*
    *     at weblogic.rjvm.ServerURL.findOrCreateRJVM(ServerURL.java:153)*
    *     at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:353)*
    *     at weblogic.jndi.Environment.getContext(Environment.java:315)*
    *     at weblogic.jndi.Environment.getContext(Environment.java:285)*
    *     at weblogic.jndi.WLInitialContextFactory.getInitialContext(WLInitialContextFactory.java:117)*
    *     at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)*
    *     at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288)*
    *     at javax.naming.InitialContext.init(InitialContext.java:223)*
    *     at javax.naming.InitialContext.<init>(InitialContext.java:197)*
    *     at weblogic.management.remote.common.ClientProviderBase.makeConnection(ClientProviderBase.java:178)*
    *     at weblogic.management.remote.common.ClientProviderBase.newJMXConnector(ClientProviderBase.java:84)*
    *     at javax.management.remote.JMXConnectorFactory.newJMXConnector(JMXConnectorFactory.java:338)*
    *     at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:247)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.getDomainMBeanServerConnection(RegisterWithDomainRuntimeService.java:222)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.notifyDomainRuntime(RegisterWithDomainRuntimeService.java:137)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.start(RegisterWithDomainRuntimeService.java:91)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeServiceLate.start(RegisterWithDomainRuntimeServiceLate.java:17)*
    *     at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)*
    *     at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)*
    *     at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)*
    *     at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)*
    Caused By: java.rmi.ConnectException: Destination unreachable; nested exception is:
    *     java.net.ConnectException: Connection refused; No available router to destination*
    *     at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:470)*
    *     at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:321)*
    *     at weblogic.rjvm.RJVMManager.findOrCreateRemoteInternal(RJVMManager.java:254)*
    *     at weblogic.rjvm.RJVMManager.findOrCreate(RJVMManager.java:197)*
    *     at weblogic.rjvm.RJVMFinder.findOrCreateRemoteServer(RJVMFinder.java:238)*
    *     at weblogic.rjvm.RJVMFinder.findOrCreateInternal(RJVMFinder.java:200)*
    *     at weblogic.rjvm.RJVMFinder.findOrCreate(RJVMFinder.java:170)*
    *     at weblogic.rjvm.ServerURL.findOrCreateRJVM(ServerURL.java:153)*
    *     at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:353)*
    *     at weblogic.jndi.Environment.getContext(Environment.java:315)*
    *     at weblogic.jndi.Environment.getContext(Environment.java:285)*
    *     at weblogic.jndi.WLInitialContextFactory.getInitialContext(WLInitialContextFactory.java:117)*
    *     at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)*
    *     at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288)*
    *     at javax.naming.InitialContext.init(InitialContext.java:223)*
    *     at javax.naming.InitialContext.<init>(InitialContext.java:197)*
    *     at weblogic.management.remote.common.ClientProviderBase.makeConnection(ClientProviderBase.java:178)*
    *     at weblogic.management.remote.common.ClientProviderBase.newJMXConnector(ClientProviderBase.java:84)*
    *     at javax.management.remote.JMXConnectorFactory.newJMXConnector(JMXConnectorFactory.java:338)*
    *     at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:247)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.getDomainMBeanServerConnection(RegisterWithDomainRuntimeService.java:222)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.notifyDomainRuntime(RegisterWithDomainRuntimeService.java:137)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeService.start(RegisterWithDomainRuntimeService.java:91)*
    *     at weblogic.management.mbeanservers.runtime.internal.RegisterWithDomainRuntimeServiceLate.start(RegisterWithDomainRuntimeServiceLate.java:17)*
    *     at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)*
    *     at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)*
    *     at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)*
    *     at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)*
    *>*
    Cheers,
    Varun

    hi,
    I was able to resolve this issue. I haven't given Listen IP address for Admin Server as I have selected All Local Addresses during domain creation. As per some blog this allows Managed Server on remote machine to use any of the Address to speak to Admin Server. In this case it is using 192.168.122.1 rather than so called IP Address.
    Simple fix will be giving DNS Name of Machine or IP Address.
    Cheers,
    Varun

Maybe you are looking for