Cluster Info / Issue ...

Hi,
I'm seeing the below log information of particular weblogic application in PRODUCTION , but i when verified the weblogic servers they were UP & RUNNING..
I don't understand what does these log says... Can someone explain me here ??
Thanks for you help as always..
plseweblogic04
####<May 20, 2010 2:49:33 AM EDT> <Info> <Cluster> <plseweblogic04> <eesServer2> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1274338173411> <BEA-000144> <Managed server eesServer1 has been suspended or shutdown.>
plseweblogic03
####<May 20, 2010 2:53:34 AM EDT> <Info> <Cluster> <plseweblogic03> <eesServer1> <[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1274338414170> <BEA-000144> <Managed server eesServer2 has been suspended or shutdown.>
Other information for clarity:
On plseweblogic04 box , we have
1) AdminServer and ManagedServer2
On plseweblogic03 box , we have
ManagedServer1

This is actually happenning due to a EOFException:
weblogic.rjvm.PeerGoneException: ; nested exception is:
java.io.EOFException
at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:234)
at weblogic.rmi.internal.BasicRemoteRef.invoke(BasicRemoteRef.java:223)
at weblogic.servlet.internal.ROIDLookupImpl_1030_WLStub.lookupROID(Unknown Source)
at weblogic.servlet.internal.HttpServer$Replicator.lookupROID(HttpServer.java:584)
And there are, in fact, NO network issues between the two servers on the same physical layer 3 subnet with no L3 network devices (such as firewalls) in the middle... It is using UNICAST cluster communications...
--Randall                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

Similar Messages

  • Routing and Remote access can cause cluster network issues?

    After enabling routing and remote access on the servers, we found lots of cluster issues on our server like<o:p></o:p>
    Cluster Service stopped
    Communication was lost and reestablished between cluster nodes
    Unable to access witness resource
    Cluster resource failed
    can RRAS enabling causes cluster network issues?
    Rahul

    Hi TwoR,
    Please offer more information about your current cluster and RRAS configuration, such as are you installed the RRAS role on any cluster node? Are your cluster in Hyper-V environment?
    Or if you want to create the RRAS cluster you can refer the following KB:
    Deploy Remote Access in a Cluster
    http://technet.microsoft.com/en-us/library/jj134175.aspx
    How to configure Network Load Balancing (NLB) based cluster of VPN Servers
    http://blogs.technet.com/b/rrasblog/archive/2009/07/02/configuring-network-load-balancing-nlb-cluster-of-vpn-servers.aspx
    I’m glad to be of help to you!
    We
    are trying to better understand customer views on social support experience, so your participation in this
    interview project would be greatly appreciated if you have time.
    Thanks for helping make community forums a great place.

  • File Share Cluster Transfer Issues - Windows 2012 R2 - Two Nodes

    Hi There,
    I am having some issues in creating a simple file share on a 2 node cluster.
    Each "File Server" is Windows 2012 R2 Standard.
    Each server has 4 Nic Ports, 2 are configured as a Control Team and the other 2 as a Media Team.  The media team is specifically for any storage and file sharing traffic, the control team is purely for access to the server via RDC.
    The storage is from a NetApp SAN connected via SAS cables to the 2 "File Servers" and transfer speeds to and from that are good (1GB/s)
    I have looked at different step-by-steps to get this working, they all are similar to: https://robertsmit.wordpress.com/2012/06/26/clustering-fileserver-on-windows-2012-step-by-step/
    The share permissions are configured with Everyone with Full Control.
    Once the file shares are all set up, I can browse the shares and any current folders and files.
    I can download any files that exist on these shares...
    The issue is when I try to put a file on the share from a "client" server (win 2008 r2) or even from the other node (connecting to the file share location as I would on another machine)
    I just get the copying dialogue attached, which sits there for about 3-4 minutes and then just finishes with a Network Error.
    This only occurs during an upload of a file to the share.
    All servers can ping each other, there are no cluster event errors..
    Can anyone help, I need to get this fixed ASAP.
    If you need more info then please ask.
    Thanks,
    -Tim

    Hi Tim,
    Did you run the cluster validation before you create this cluster? Please disable any firewall or AV soft and install the recommended hotfixes and updates for Windows Server
    2012 R2-based failover clusters update to narrow down the issue first.
    Recommended hotfixes and updates for Windows Server 2012 R2-based failover clusters
    http://support.microsoft.com/kb/2920151
    I’m glad to be of help to you!
    Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact [email protected]

  • BI Publisher Cluster Deployment Issue

    Hi All,
    I have installed BI Publisher in two nodes A and B (stand alone basic). During scheduler configuration, I have checked cluster option. A load balancer has been configured to point to these two nodes.The data source is file system and location is on network file server.
    I have a java program that call BI Publisher webservice and submit the job for scheduling. The java program points to load balancer ip. Currently I am facing two issues as below:
    1. When I call BI Publisher webservice to schedule the job, I can see that the job is scheduled through one node only
    2. Processing the scheduled job (report generation is distributed into two nodes. But this distribution is not even. If I shedule 5 jobs, Node A is executing one job while the other node executes reamining 4 jobs. Thus I have noticed that Node A remains idle after completing the job till the other node completes the report generation
    Do I need to do make any changes/ perform additional configuration so that the job is distributed among the nodes evenly.
    Your help will be highlu appreciated
    Thanks
    Angshuman

    First, make sure you've got enough machine resources as BIEE11g is resources hungry.
    Next, after installation you should have icons to start BIEE11g (try windows version as it is easier to start up).
    Run the startBIEE11g script and it will take some minutes, be patient. Once you get all processes running
    then you can try to login.
    Take a look at my post:
    http://oraclebiblog.blogspot.com/2010/11/starting-and-stopping-biee-from-command.html
    Good luck
    regards
    Jorge

  • SAP Cluster service issue

    Here is the description of the PRD cluster scenario. ( windows 2008 + oracle)
    We have 2 nodes .
    1. host-erpn01 ( Have ASCS , Database instance, Enqueue and Dialog
    Instance installed)
    2. host-erp02 ( Have Central Instance, Dialog Instance and Enqueue installed)
    When we move "SAP SID" service using "failover cluster management tool" from one node to another its fails and we have to manually select the  "SAP SID cluster service" and "SAP SID cluster instance" to online.
    These both service and instance were coming online after manual selection, however after some time in the mmc console of node 2 the sap instances hosted on node1 are in red cross and are giving " cannot connect to sap service dcom interface error 800706BA"
    We replaced the sapstartsrv.exe from working directory of ASCS instance to CI executable directory.
    Now the disp+work is stopped for CI instance. Also in the CI instance executable directory we can see five files with name of sapstartsrv i.e
    sapstartsrv.exe.new , sapstartsrv.exe.tmp, sapstartsrv.new, sapstartsrv.pdb and actual sapstartsrv.exe file.
    Here is the log of sapstartsrv.log  CI work directory from node2.
    trc file: "sapstartsrv.log", trc level: 0, release: "701"
    pid        1968
    Mon Oct 11 15:55:33 2010
    SAP HA Trace: Build in SAP Microsoft Cluster library '701, patch 32, changelist 1046543' initialized
    Initializing SAPControl Webservice
    SapSSLInit failed => https support disabled
    Starting WebService Named Pipe thread
    Starting WebService thread
    Webservice named pipe thread started, listening on port
    .\pipe\sapcontrol_01
    Webservice thread started, listening on port 50113
    GCCIA\csrvadmin is starting SAP System at 2010/10/11 16:09:07
    SAP HA Trace: FindClusterResource: SAP resource not found [sapwinha.cpp, line 334]
    SAP HA Trace: SAP_HA_FindSAPInstance returns: SAP_HA_NOT_CLUSTERED [sapwinha.cpp, line 907]"
    or you can view other logs from the work directory dump at
    http://s000.tinyupload.com/index.php?file_id=45384422007535688902
    Now when we try to start the SAPSID_00 service manually its giving error "The SAPSID_00 service failed to start due to the following error: The system cannot find the path specified.
    Please advice.
    Regards
    Edited by: Tech GCCIA on Oct 11, 2010 3:27 PM
    Edited by: Tech GCCIA on Oct 11, 2010 3:28 PM

    Hi Sunil ,
                       On node 1 there is no  listener.trc at /oracle_home/network/trace folder , here is the log of listener.log file in case if it is helpful.
    TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 10:37:37
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    System parameter file is D:\oracle\GCP\102\network\admin\listener.ora
    Log messages written to D:\oracle\GCP\102\network\log\listener.log
    Trace information written to D:\oracle\GCP\102\network\trace\listener.trc
    Trace level is currently 0
    Started with pid=3116
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCP.WORLDipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCPipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=gccia-erpn01.gccia.com.sa)(PORT=1527)))
    Listener completed notification to CRS on start
    TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE
    TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 11:59:37
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    System parameter file is D:\oracle\GCP\102\network\admin\listener.ora
    Log messages written to D:\oracle\GCP\102\network\log\listener.log
    Trace information written to D:\oracle\GCP\102\network\trace\listener.trc
    Trace level is currently 0
    Started with pid=5036
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCP.WORLDipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCPipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))
    Listener completed notification to CRS on start
    TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE
    10-OCT-2010 12:00:31 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=60592)) * establish * GCP * 0
    10-OCT-2010 12:00:31 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=60593)) * establish * GCP * 0
    10-OCT-2010 12:00:31 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=60594)) * establish * GCP * 0
    10-OCT-2010 12:00:31 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=60595)) * establish * GCP * 0
    10-OCT-2010 12:00:31 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=60596)) * establish * GCP * 0
    10-OCT-2010 13:01:19 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61336)) * establish * GCP * 0
    10-OCT-2010 13:01:37 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61340)) * establish * GCP * 0
    10-OCT-2010 13:01:37 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61341)) * establish * GCP * 0
    10-OCT-2010 13:01:37 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61342)) * establish * GCP * 0
    10-OCT-2010 13:01:37 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61343)) * establish * GCP * 0
    10-OCT-2010 13:01:37 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61344)) * establish * GCP * 0
    10-OCT-2010 13:08:27 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61485)) * establish * GCP * 0
    10-OCT-2010 13:08:42 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61489)) * establish * GCP * 0
    10-OCT-2010 13:08:42 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61490)) * establish * GCP * 0
    10-OCT-2010 13:08:42 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61491)) * establish * GCP * 0
    10-OCT-2010 13:08:42 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61492)) * establish * GCP * 0
    10-OCT-2010 13:08:42 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61493)) * establish * GCP * 0
    TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 13:09:57
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    System parameter file is D:\oracle\GCP\102\network\admin\listener.ora
    Log messages written to D:\oracle\GCP\102\network\log\listener.log
    Trace information written to D:\oracle\GCP\102\network\trace\listener.trc
    Trace level is currently 0
    Started with pid=2336
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCP.WORLDipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCPipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))
    Listener completed notification to CRS on start
    TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE
    TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 13:14:34
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    System parameter file is D:\oracle\GCP\102\network\admin\listener.ora
    Log messages written to D:\oracle\GCP\102\network\log\listener.log
    Trace information written to D:\oracle\GCP\102\network\trace\listener.trc
    Trace level is currently 0
    Started with pid=4948
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCP.WORLDipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCPipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))
    Listener completed notification to CRS on start
    TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE
    TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 13:38:12
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    System parameter file is D:\oracle\GCP\102\network\admin\listener.ora
    Log messages written to D:\oracle\GCP\102\network\log\listener.log
    Trace information written to D:\oracle\GCP\102\network\trace\listener.trc
    Trace level is currently 0
    Started with pid=2456
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCP.WORLDipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCPipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))
    Listener completed notification to CRS on start
    TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE
    TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 14:03:35
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    System parameter file is D:\oracle\GCP\102\network\admin\listener.ora
    Log messages written to D:\oracle\GCP\102\network\log\listener.log
    Trace information written to D:\oracle\GCP\102\network\trace\listener.trc
    Trace level is currently 0
    Started with pid=2756
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCP.WORLDipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCPipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))
    Listener completed notification to CRS on start
    TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE
    TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 14:10:42
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    System parameter file is D:\oracle\GCP\102\network\admin\listener.ora
    Log messages written to D:\oracle\GCP\102\network\log\listener.log
    Trace information written to D:\oracle\GCP\102\network\trace\listener.trc
    Trace level is currently 0
    Started with pid=4812
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCP.WORLDipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
    .\pipe\GCPipc)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))
    Listener completed notification to CRS on start
    TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE
    TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 11-OCT-2010 09:34:05
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    System parameter file is D:\oracle\GCP\102\network\admin\listener.ora
    Log messages written to D:\oracle\GCP\102\network\log\listener.log
    Trace information written to D:\oracle\GCP\102\network\trace\listener.trc
    Trace level is currently 0
    Started with pid=1920
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))
    Listener completed notification to CRS on start
    TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE
    TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 11-OCT-2010 21:12:29
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    System parameter file is D:\oracle\GCP\102\network\admin\listener.ora
    Log messages written to D:\oracle\GCP\102\network\log\listener.log
    Trace information written to D:\oracle\GCP\102\network\trace\listener.trc
    Trace level is currently 0
    Started with pid=1952
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))
    Listener completed notification to CRS on start
    TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE

  • FTP Adapter  Sync Read in SOA Suite cluster  giving issues

    Hello All ,
    I am trying to use a FTP Adapter in sync read mode in bpel in clustered environment.
    FTP Adapter has to sync read a file from a remote unix server. And during this the error that comes is file not found.
    ORABPEL-11007
    No Data to process.
    File /o31364629.pdf to be processed was not found or not available or has no content
    Whereas FILE IS THERE IN THE DIRECTORY.
    This issue is coming intermittent . I do have some successful BPEL instances coming up.
    But majorly its failing.
    Now turnign the logs in DEBUG mode said its trying to create some temp file which is unable.
    Caused by: ORABPEL-11440
    Error in writing temp file after getting content.
    Error in writing temp file after getting content for file: o31364629.pdf
    Please ensure 1. the local filesystem has write permission 2. local file system is not full.
    Please see below full log.
    <2009-12-16 10:47:30,970> <INFO> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> FTPInteraction Created
    <2009-12-16 10:47:30,971> <DEBUG> <LiebertServices.collaxa.cube.ws> <AdapterFramework::Outbound> file:/apps/erikap1/aerikap1/erikap1_SoaSuite/bpel/domains/LiebertServices/tmp/.bpel_Test_FTP_1.0_c904a18cefb4038ec3b754d37b34d2d1.tmp/New_FTP_adap.wsdl [ SynchRead_ptt::SynchRead(Empty,opaque) ] - Starting JCA LocalTransaction
    <2009-12-16 10:47:30,971> <DEBUG> <LiebertServices.collaxa.cube.ws> <AdapterFramework::Outbound> file:/apps/erikap1/aerikap1/erikap1_SoaSuite/bpel/domains/LiebertServices/tmp/.bpel_Test_FTP_1.0_c904a18cefb4038ec3b754d37b34d2d1.tmp/New_FTP_adap.wsdl [ SynchRead_ptt::SynchRead(Empty,opaque) ] - Invoking JCA Outbound Interaction
    <2009-12-16 10:47:30,971> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> FTP outbound adapter interaction invoked : oracle.tip.adapter.ftp.outbound.FTPReadInteractionSpec@17b7cfb
    <2009-12-16 10:47:30,971> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> FileInteraction::PROPS ====>{}
    <2009-12-16 10:47:30,972> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> Considering header for filename, output filename from header: o31364629.pdf
    <2009-12-16 10:47:30,972> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> Not Considering header for directory as it is blank
    <2009-12-16 10:47:30,972> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> FTP Agent Determined
    <2009-12-16 10:47:30,972> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> FTPSource::Local Temp Directory is /ftpdata/elshap1/reach/fileftp/temp_ftp/localhost_LiebertServices_Test_FTP~1.0_/outbound
    <2009-12-16 10:47:30,972> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> FTP Host: dianadb.emrsn.com
    <2009-12-16 10:47:30,972> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> FTP Post: 21
    <2009-12-16 10:47:30,973> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> FTP Username: elsha_p
    <2009-12-16 10:47:30,973> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> FTP Keep Connections: true
    <2009-12-16 10:47:30,973> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> FTP outbound adapter read file interaction invoked.
    <2009-12-16 10:47:30,973> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> Outbound File Adapter received request to read file with parameters: { Directory=, FileName=o31364629.pdf, DeleteFile=false}, headers=[_directory=[], _filename=[o31364629.pdf], _props=[{}]]
    <2009-12-16 10:47:30,974> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> Creating Temp File iwCmE1Jr7hUYMdXRISl0OA== Local Path: /ftpdata/elshap1/reach/fileftp/temp_ftp/localhost_LiebertServices_Test_FTP~1.0_/outbound Remote Dir:
    <2009-12-16 10:47:30,976> <DEBUG> <LiebertServices.collaxa.cube.ws> <File Adapter::Outbound> Calling getContentsAsStream() on FTPAgent: dianadb.emrsn.com: type = FTP, host = dianadb.emrsn.com:21, username = elsha_p
    directory = null, mode = STREAM, fileType = ASCII, maxRetries = 0 Localfile: o31364629.pdf Path: /ftpdata/elshap1/reach/fileftp/temp_ftp/localhost_LiebertServices_Test_FTP~1.0_/outbound/iwCmE1Jr7hUYMdXRISl0OA== Remote Dir:_
    <2009-12-16 10:47:30,982> <ERROR> <LiebertServices.collaxa.cube.ws> <AdapterFramework::Outbound> file:/apps/erikap1/aerikap1/erikap1_SoaSuite/bpel/domains/LiebertServices/tmp/.bpel_Test_FTP_1.0_c904a18cefb4038ec3b754d37b34d2d1.tmp/New_FTP_adap.wsdl [ SynchRead_ptt::SynchRead(Empty,opaque) ] - Could not invoke operation 'SynchRead' against the 'FTP Adapter' due to:
    ORABPEL-11007
    No Data to process.
    File /o31364629.pdf to be processed was not found or not available or has no content
    Check the error stack and fix the cause of the error. Contact oracle support if error is not fixable.
         at oracle.tip.adapter.file.outbound.FileReader.readFile(FileReader.java:253)
         at oracle.tip.adapter.file.outbound.FileReader.executeFileRead(FileReader.java:168)
         at oracle.tip.adapter.file.outbound.FileInteraction.executeFileRead(FileInteraction.java:248)
         at oracle.tip.adapter.ftp.outbound.FTPInteraction.execute(FTPInteraction.java:172)
         at oracle.tip.adapter.fw.wsif.jca.WSIFOperation_JCA.executeRequestResponseOperation(WSIFOperation_JCA.java:486)
         at com.collaxa.cube.ws.WSIFInvocationHandler.invoke(WSIFInvocationHandler.java:479)
         at com.collaxa.cube.ws.WSInvocationManager.invoke2(WSInvocationManager.java:528)
         at com.collaxa.cube.ws.WSInvocationManager.invoke(WSInvocationManager.java:285)
         at com.collaxa.cube.engine.ext.wmp.BPELInvokeWMP.__invoke(BPELInvokeWMP.java:829)
         at com.collaxa.cube.engine.ext.wmp.BPELInvokeWMP.__executeStatements(BPELInvokeWMP.java:412)
         at com.collaxa.cube.engine.ext.wmp.BPELActivityWMP.perform(BPELActivityWMP.java:199)
         at com.collaxa.cube.engine.CubeEngine.performActivity(CubeEngine.java:3714)
         at com.collaxa.cube.engine.CubeEngine.handleWorkItem(CubeEngine.java:1657)
         at com.collaxa.cube.engine.dispatch.message.instance.PerformMessageHandler.handleLocal(PerformMessageHandler.java:75)
         at com.collaxa.cube.engine.dispatch.DispatchHelper.handleLocalMessage(DispatchHelper.java:220)
         at com.collaxa.cube.engine.dispatch.DispatchHelper.sendMemory(DispatchHelper.java:317)
         at com.collaxa.cube.engine.CubeEngine.endRequest(CubeEngine.java:5787)
         at com.collaxa.cube.engine.CubeEngine.createAndInvoke(CubeEngine.java:1089)
         at com.collaxa.cube.engine.delivery.DeliveryService.handleInvoke(DeliveryService.java:589)
         at com.collaxa.cube.engine.ejb.impl.CubeDeliveryBean.handleInvoke(CubeDeliveryBean.java:421)
         at sun.reflect.GeneratedMethodAccessor174.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:585)
         at com.evermind.server.ejb.interceptor.joinpoint.EJBJoinPointImpl.invoke(EJBJoinPointImpl.java:35)
         at com.evermind.server.ejb.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:119)
         at com.evermind.server.ejb.interceptor.system.DMSInterceptor.invoke(DMSInterceptor.java:52)
         at com.evermind.server.ejb.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:119)
         at com.evermind.server.ejb.interceptor.system.JAASInterceptor$1.run(JAASInterceptor.java:31)
         at com.evermind.server.ThreadState.runAs(ThreadState.java:693)
         at com.evermind.server.ejb.interceptor.system.JAASInterceptor.invoke(JAASInterceptor.java:34)
         at com.evermind.server.ejb.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:119)
         at com.evermind.server.ejb.interceptor.system.TxRequiredInterceptor.invoke(TxRequiredInterceptor.java:50)
         at com.evermind.server.ejb.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:119)
         at com.evermind.server.ejb.interceptor.system.DMSInterceptor.invoke(DMSInterceptor.java:52)
         at com.evermind.server.ejb.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:119)
         at com.evermind.server.ejb.InvocationContextPool.invoke(InvocationContextPool.java:55)
         at com.evermind.server.ejb.StatelessSessionEJBObject.OC4J_invokeMethod(StatelessSessionEJBObject.java:87)
         at CubeDeliveryBean_LocalProxy_4bin6i8.handleInvoke(Unknown Source)
         at com.collaxa.cube.engine.dispatch.message.invoke.InvokeInstanceMessageHandler.handle(InvokeInstanceMessageHandler.java:37)
         at com.collaxa.cube.engine.dispatch.DispatchHelper.handleMessage(DispatchHelper.java:141)
         at com.collaxa.cube.engine.dispatch.BaseDispatchTask.run(BaseDispatchTask.java:58)
         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
         at java.lang.Thread.run(Thread.java:595)
    Caused by: ORABPEL-11440
    Error in writing temp file after getting content.
    Error in writing temp file after getting content for file: o31364629.pdf
    Please ensure 1. the local filesystem has write permission 2. local file system is not full.
         at oracle.tip.adapter.ftp.inbound.FTPSource.getContents(FTPSource.java:883)
         at oracle.tip.adapter.ftp.inbound.FTPSource.getFileFromSource(FTPSource.java:799)
         at oracle.tip.adapter.file.outbound.FileReader.readFile(FileReader.java:250)
         ... 43 more
    I am using the setting for file adapter to avoid race condition in HA environment.
    Set oracle.tip.adapter.file.controldirpath=/ftpdata/elshap1/reach in pc.properties file
    This is shared folder accesible from both nodes & is given 777 full permission till the folder outbound where the temp fiel is getting created.
    Please let me know if you have any idea on this.
    This is a severe prodcution issue for us.
    thanks in advance.
    --Shivani                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       

    Just want to highlight the below portion in the log listed.
    Caused by: ORABPEL-11440
    Error in writing temp file after getting content.
    Error in writing temp file after getting content for file: o31364629.pdf
    Please ensure 1. the local filesystem has write permission 2. local file system is not full.
         at oracle.tip.adapter.ftp.inbound.FTPSource.getContents(FTPSource.java:883)
         at oracle.tip.adapter.ftp.inbound.FTPSource.getFileFromSource(FTPSource.java:799)
         at oracle.tip.adapter.file.outbound.FileReader.readFile(FileReader.java:250)
         ... 43 more
    Thanks Shivani

  • Oracle10g RAC Cluster Interconnect issues

    Hello Everybody,
    Just a brief overview as to what i am currently doing. I have installed Oracle10g RAC database on a cluster of two Windows 2000 AS nodes.These two nodes are accessing an external SCSI hard disk.I have used Oracle cluster file system.
    Currently i am facing some performance issues when it comes to balancing workload on both the nodes.(Single instance database load is faster than a parallel load using two database instances).
    I feel the performance issues could be due to IPC using public Ethernet IP instead of private interconnect.
    (During a parallel load large amount of packets of data are sent over the Public IP and not Private interconnect).
    How can i be sure that the Private interconnect is used for transferring cluster traffic and not the Public IP? (Oracle mentions that for a Oracle10g RAC database, private IP should be used for heart beat as well as transferring cluster traffic).
    Thanks in advance,
    Regards,
    Salil

    You find the answers here:
    RAC: Frequently Asked Questions
    Doc ID: NOTE:220970.1
    At least crossover interconnect is completely unsupported.
    Werner

  • Windows 2008r2 cluster dns issue

    We are running GW 8.0.2 not hot patches. System is running on a windows 2008r2 2 node cluster. When we failover the gwia from node 1 to node 2 we can no longer send email to the internet. We receive just fine, also imap and pop connections are working. We have checked firewall settings and they are the same for both nodes. We get 450 hosts unknown errors for all email going out. We think this is a dns issue, but can't find where the problem may be. We have checked the registry dns nameserver entries on both nodes and they match.
    Thanks,
    Bill

    On the GWIA, is the bind exclusive set on the addressing tab?
    The GWIA will send on the primary address of the box unless this box
    is checked - but it will listen on all IP addresses (this inbound will
    work but recieve will not)
    T
    On Fri, 01 Apr 2011 12:38:04 GMT, [email protected]: wrote:
    >We are running GW 8.0.2 not hot patches. System is running on a windows 2008r2 2 node cluster. When we failover the gwia from node 1 to node 2 we can no longer send email to the internet. We receive just fine, also imap and pop connections are working. We have checked firewall settings and they are the same for both nodes. We get 450 hosts unknown errors for all email going out. We think this is a dns issue, but can't find where the problem may be. We have checked the registry dns nameserver entries on both nodes and they match.
    >
    >Thanks,
    >Bill

  • BAM Cluster setup - issue with whole server migration configuration

    HI,
    We are working on a SOA , BAM , BPM Cluster setup on OEL VMs
    With Admin server running on Host1 and Soa_server1, BAM_server1 , Host2 - Soa_server2 , Bam_server2.
    We were successfully able to setup clustering for Soa without using the webTier utilities in our environment, Just by adding entries in the /etc/hosts file.
    Followed the document - except for web tier setup
    http://www.namredlek.nl/orasoa/InstallSOASuite11gOnCluster_v1.pdf
    1)     Installed Weblogic 11g.
    2)     Soa 11g.
    3)     Domain creation
    4) Configured Node manager
    5) Soa Clustering completed successfully.
    6) Domain up and running with SOA_Cluster up and running.
    7)     Extended the domain with BAM
    The issue revolves when we are doing teh Migration configurations from the console for BAM and we restart BAM server we get an error -
    Apr 14, 2011 2:30:35 AM weblogic.nodemanager.server.Handler handleStart
    WARNING: Exception while starting server 'bam_server1'
    java.io.IOException: Command '/oracle/Shared_MWHome/user_projects/domains/SOA_Domain/bin/server_migration/wlsifconfig.sh -addif eth0 192.168.1.6 255.255.255.0 ' returned an unsuccessful exit code '1'. Check NM logs for script output.
    and Bam server goes to Failed Not Restartable state.

    The error is clearly out there
    java.io.IOException: Command '/oracle/Shared_MWHome/user_projects/domains/SOA_Domain/bin/server_migration/wlsifconfig.sh -addif eth0 192.168.1.6 255.255.255.0 ' returned an unsuccessful exit code '1'. Check NM logs for script output.
    This error occurs due to a failure in adding the virtual IP of BAM server to the network interface you have specified in nodemanager.properties.
    You need to check whether you have specified the correct interface and subnet mask in node manager.properties as below :
    The fourth step is to edit the Node Manager's properties file. This file is called nodemanager.properties
    and is located in the ORACLE_BASE/product/osbmw/wlserver_10.3/common/nodemanager directory.
    For server migration to work properly, you need to add the properties listed below:
    Interface=eth0
    NetMask=255.255.255.0
    UseMACBroadcast=true
    · Interface
    This property specifies the interface name for the floating IP (for example, eth0).
    · NetMask
    This property specifies the net mask for the interface for the floating IP. The net mask should the
    same as the net mask on the interface; 255.255.255.0 is used as an example in this document.
    · UseMACBroadcast
    This property specifies whether or not to use a node's MAC address when sending ARP packets,
    that is, whether or not to use the -b flag in the arping command.

  • EXIF Info issue with Teleconverters

    Hello everybody,
    I think I have a problem with EXIF in Aperture (version 3.4.3) on my MacBook Pro (OS 10.8.3).
    For photos taken with Canon lenses plus 1.4/2.0 Canon teleconverters both Camera Info and EXIF Info are displaing all the information correctly except for the TC which is missinig. The "+1.4x/+2.0x" is not displayed after the lens model name which, as I said, is correct. This wasn't happening in the past; older photos in the same library are still showing all the proper information.
    However opening raw files with Preview, under the Canon tab in the More Info Inspector, I can see the name of the lens + TC. I've checked this both for a "new" photo (not working properly in Aperture) and an "old" one (ok in Aperture); therefore I think it's not an error within the raw files.
    Does anyone have an idea for that and how to fix it?
    Thank you very much.

    Dear Umang,
    It is very clear that when Condiyion Type is not maintained in VD52, there is no error. Ideally, Condition Type is not to be maintained in VD52, to avoid such issues.
    Hence it is advisable to convince the client that it is not recommended practise to maintain Condition Type in VD52.
    It is also recommended that along with a Abaper, debug and try to identify the root cause.
    It is also possible that Condition type Maintained in VD52 and in Pricing Procedure is different.
    Regards,
    Rajesh Banka

  • Adobe Cluster Locking issues

    We have two nodes in the cluster. Adobe is installed on just one node. Locator is running on one node. Experiencing locking issues for temporary diskstore. Below is the error description.
    [12/10/14 13:13:44:925 EST] 00000013 webapp   
    I com.ibm.ws.webcontainer.webapp.WebGroupImpl WebGroup SRVE0169I: Loading Web Module: dsc.war.
    [12/10/14 13:13:44:935 EST] 00000013 WASSessionCor I SessionContextRegistry getSessionContext SESN0176I: Will create a new session context for application key default_host/dsc
    [12/10/14 13:13:44:963 EST] 00000013 ServletWrappe I com.ibm.ws.webcontainer.servlet.ServletWrapper init SRVE0242I: [LiveCycleES4] [/dsc] [UploadServlet]: Initialization successful.
    [12/10/14 13:13:44:970 EST] 00000013 ArchiveFileMa A   ArchiveFileManagerImpl:getLocalCacheRootDir():Adobe tmp directory is:/data/exigen_test/PT/jvm/tmp/adobews__-581654660
    [12/10/14 13:14:17:041 EST] 00000013 LocalExceptio E   CNTR0020E: EJB threw an unexpected (non-declared) exception during invocation of method "getDomain" on bean "BeanId(LiveCycleES4#adobe-pof.jar#adobe_POFDataDictionaryLocalEJB, null)". Exception data: com.gemstone.gemfire.cache.DiskAccessException: For DiskStore: ReplicatedGuaranteed_POF_DD_CACHE: Could not lock "/data/exigen_test/PT/jvm/tmp/adobews__-581654660/Caching/Data/ReplicatedGuaranteed_POF_D D_CACHE/DRLK_IFReplicatedGuaranteed_POF_DD_CACHE.lk". Other JVMs might have created diskstore with same name using the same directory., caused by java.io.IOException: No locks available
    at com.gemstone.gemfire.internal.cache.DiskStoreImpl.createLockFile(DiskStoreImpl.java:1800)
    at com.gemstone.gemfire.internal.cache.DiskStoreImpl.loadFiles(DiskStoreImpl.java:1920)
    at com.gemstone.gemfire.internal.cache.DiskStoreImpl.<init>(DiskStoreImpl.java:355)
    at com.gemstone.gemfire.internal.cache.DiskStoreImpl.<init>(DiskStoreImpl.java:270)
    at com.gemstone.gemfire.internal.cache.DiskStoreImpl.<init>(DiskStoreImpl.java:265)
    at com.gemstone.gemfire.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java :127)
    at com.adobe.livecycle.cache.adapter.GemfireCacheAdapter.createSubRegion(GemfireCacheAdapter .java:1043)
    at com.adobe.livecycle.cache.adapter.GemfireCacheAdapter.init(GemfireCacheAdapter.java:234)
    at com.adobe.livecycle.cache.adapter.GemfireCacheAdapter.<init>(GemfireCacheAdapter.java:215 )
    at com.adobe.livecycle.cache.adapter.CacheAdapterFactory.getCache(CacheAdapterFactory.java:1 03)
    at com.adobe.pof.POFUtil.getPOFDataDictionaryCache(POFUtil.java:525)
    at com.adobe.pof.adapter.AbstractAdapterContext.getDataDictionaryCache(AbstractAdapterContex t.java:89)
    at com.adobe.pof.adapter.AbstractAdapterContext.getDomainFromCache(AbstractAdapterContext.ja va:127)
    at com.adobe.pof.adapter.JDBCAdapter.getDomain(JDBCAdapter.java:4436)
    at com.adobe.pof.schema.POFDataDictionaryImpl.getDomain(POFDataDictionaryImpl.java:113)
    at com.adobe.pof.schema.POFDataDictionaryRemoteBean.getDomain(POFDataDictionaryRemoteBean.ja va:148)
    at com.adobe.pof.schema.EJSLocalStatelessadobe_POFDataDictionaryLocalEJB_4137caa5.getDomain( Unknown Source)
    at com.adobe.pof.schema.POFDataDictionaryLocalEJBAdapter.getDomain(POFDataDictionaryLocalEJB Adapter.java:69)
    at com.adobe.pof.odapi.POFSchemaManagerRemoteBean.getDomain(POFSchemaManagerRemoteBean.java: 577)
    at com.adobe.pof.odapi.EJSLocalStatelessadobe_POFSchemaManagerLocalEJB_626825cc.getDomain(Un known Source)
    at com.adobe.pof.odapi.POFSchemaManagerLocalEJBAdapter.getDomain(POFSchemaManagerLocalEJBAda pter.java:169)
    at com.adobe.idp.dsc.startup.DSCStartupServlet$1.doInTransaction(DSCStartupServlet.java:137)
    at com.adobe.idp.dsc.transaction.impl.ejb.adapter.EjbTransactionBMTAdapterBean.doRequiresNew (EjbTransactionBMTAdapterBean.java:218)
    at com.adobe.idp.dsc.transaction.impl.ejb.adapter.EJSLocalStatelessEjbTransactionBMTAdapter_ 3af08fdf.doRequiresNew(Unknown Source)
    at com.adobe.idp.dsc.transaction.impl.ejb.EjbTransactionProvider.execute(EjbTransactionProvi der.java:133)
    at com.adobe.idp.dsc.transaction.impl.DefaultTransactionTemplate.execute(DefaultTransactionT emplate.java:79)
    at com.adobe.idp.dsc.startup.DSCStartupServlet.init(DSCStartupServlet.java:132)
    at javax.servlet.GenericServlet.init(GenericServlet.java:161)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.init(ServletWrapper.java:342)
    at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.init(ServletWrapperImpl.java:168)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.loadOnStartupCheck(ServletWrapper.java:136 6)
    at com.ibm.ws.webcontainer.webapp.WebApp.doLoadOnStartupActions(WebApp.java:618)
    at com.ibm.ws.webcontainer.webapp.WebApp.commonInitializationFinally(WebApp.java:587)
    at com.ibm.ws.webcontainer.webapp.WebAppImpl.initialize(WebAppImpl.java:421)
    at com.ibm.ws.webcontainer.webapp.WebGroupImpl.addWebApplication(WebGroupImpl.java:88)
    at com.ibm.ws.webcontainer.VirtualHostImpl.addWebApplication(VirtualHostImpl.java:169)
    at com.ibm.ws.webcontainer.WSWebContainer.addWebApp(WSWebContainer.java:749)
    at com.ibm.ws.webcontainer.WSWebContainer.addWebApplication(WSWebContainer.java:634)
    at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:422)
    at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:714)
    at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1165)
    at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedAppl icationImpl.java:1369)
    at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:639)
    at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:9 67)
    at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java: 771)
    at com.ibm.ws.runtime.component.ApplicationMgrImpl$5.run(ApplicationMgrImpl.java:2169)
    at com.ibm.ws.security.auth.ContextManagerImpl.runAs(ContextManagerImpl.java:5468)
    at com.ibm.ws.security.auth.ContextManagerImpl.runAsSystem(ContextManagerImpl.java:5594)
    at com.ibm.ws.security.core.SecurityContext.runAsSystem(SecurityContext.java:255)
    at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:2174)
    at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:389 )
    at com.ibm.ws.runtime.component.CompositionUnitImpl.start(CompositionUnitImpl.java:123)
      at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.ja va:496)
            at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1702)
    Caused by: java.io.IOException: No locks available
            at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:924)
            at java.nio.channels.FileChannel.tryLock(FileChannel.java:973)
            at com.gemstone.gemfire.internal.cache.DiskStoreImpl.createLockFile(DiskStoreImpl.java:1781)
            ... 56 more
    Any immediate help would be appreciated to resolve the issue. Thank you.

    Similar issue : https://forums.adobe.com/message/6156015#6156015
    Check the configuration of temp as well as GDS directories for the cluster : http://www.avoka.com/blog/2008/06/livecycle-directories-global-storagetemp-clustering-and- more/
    -Wasil

  • OSB 11gR1 Cluster Environment Issues

    Hi,
    I installed Oracle Service Bus 11gR1 on a clustered environment with two nodes and extended the domain for OSB. (First Node: osb_server1, Second Node: osb_server2)
    I am facing the problems below on this environment:
    1. When I login on SB Console on second node, on "Operations" menu a warning message (Unable to obtain metrics data from the server. ) is displayed and the message below is logged on osb_server2:
    <Aug 14, 2012 3:21:00 PM GMT+03:00> <Error> <ALSB Statistics Manager>
    <BEA-473003> <Aggregation Server Not Available. Failed to get remote aggregator
    java.lang.IllegalArgumentException: Server 'null' not found
    at com.bea.alsb.platform.weblogic.WlsDomainConfigurationImpl.getServer(WlsDomainConfigurationImpl.java:98)
    at com.bea.alsb.platform.weblogic.WlsDomainConfigurationImpl.getAggregationServer(WlsDomainConfigurationImpl.java:119)
    at com.bea.wli.monitoring.statistics.ALSBStatisticsManager.getRemoteAggregator(ALSBStatisticsManager.java:291)
    at com.bea.wli.monitoring.statistics.ALSBStatisticsManager.access$000(ALSBStatisticsManager.java:38)
    at com.bea.wli.monitoring.statistics.ALSBStatisticsManager$RemoteAggregatorProxy.send(ALSBStatisticsManager.java:55)
    Truncated. see log file for complete stacktrace
    >
    2. In SB Console on both nodes when "Security" menu is selected, an error message (Message The server encountered an unexpected condition which prevented it from fulfilling the request. ) is displayed and the message (
    <Aug 14, 2012 3:16:03 PM EEST> <Error> <User-Management-WLI-OAM> <BEA-000000>
    <[BEA-WLI-Security-UserManagement:482200]The operation searchUsers is not supported by the provider null for user/group .>
    <Aug 14, 2012 3:16:03 PM EEST> <Error> <User-Management-WLI-OAM> <BEA-000000>
    <[BEA-WLI-Security-UserManagement:482200]The operation searchUsers is not supported by the provider null for user/group .>
    <Aug 14, 2012 3:16:03 PM EEST> <Error> <ALSB Console> <BEA-494002> <Internal error occured in OSBConsole : null
    com.bea.p13n.security.management.OperationNotSupportedException
    at com.bea.p13n.security.management.authentication.AtnManagerProxy.getUserNames(AtnManagerProxy.java:579)
    at com.bea.alsb.console.oam.usermanagement.UserManagementHelper.getUserNames(UserManagementHelper.java:857)
    at com.bea.alsb.console.oam.usermanagement.UserManagementHelper.searchUsers(UserManagementHelper.java:138)
    at com.bea.alsb.console.usermanagement.user.UserManagement.viewUsersByName(UserManagement.java:244)
    at com.bea.alsb.console.usermanagement.user.UserManagement.viewUsers(UserManagement.java:71)
    Truncated. see log file for complete stacktrace
    >
    How can I solve these two issues?

    For the first error - Refer to this http://tim.blackamber.org.uk/?p=975
    For the second error - Looks like there is an issue loading the default ldap configuration. Please do the following:
    1. Take a backup of the following folders cache+, data+ and tmp+ in each of the servers.
    2. Then delete the contents of the above folders in each server
    3. And restart all the servers in the cluster. This should reinitialize all the required files at startup.
    Hope this helps.
    Thanks,
    Patrick

  • Sql2012 multisite cluster connection issues

    Hello,
    I have a two node multisite sql2012 cluster.  This cluster resides in domainA and the clients in domainA are connecting without issue.  We have a second Domain we will call DomainB that also needs to connect to the sql cluster. We do not have a
    trust relationship between the domains established nor are we replicating dns between the two domains. What I did in DomainB was add two host address records. For example:
    Cluster name - SQLClus
    DNS Records
    SQLClus - 10.210.200.233  (Node 1)
    SQLClus - 10.210.201.233  (Node 2)
    This is how things are setup in DomainA and it works without issue. For connectivity I am using the sql native client 11 in both domains.  The only thing that I can think of that is different is the fact that in DomainA the nodes are members of that
    domain and they are setup to register their dns records. In DomainB the dns records are just static records that I added.
    To provide futher explanation of the issue when I am trying to connect from DomainB the connection fails on occasion and when it fails if I ping the cluster name it is going to secondary server which is not active at the time. I thought the native client
    ODBC driver was supposed to be able to handle this and it would try the second IP listed in dns?
    Any thoughts on how to get this working properly across the two different domains? 

    Unfortunately, nodes in a Windows Server Failover Cluster (WSFC) need to be in the same Active Directory domain even when they are in different network subnet. Check the section
    Additional Considerations for Multi-Subnet Configurations from this MSDN article
    http://msdn.microsoft.com/en-us/library/ms189910.aspx
    The WSFC should be the one creating the DNS records via the Active Directory integration. Otherwise, the cluster would not be able to work with the DNS records
    Edwin Sarmiento SQL Server MVP | Microsoft Certified Master
    Blog |
    Twitter | LinkedIn
    SQL Server High Availability and Disaster Recover Deep Dive Course

  • Cluster Info

    Hi,
    i am getting below INFO and errors in logs i want to what each of them is occuring, can any one help me in my understanding.
    ####<Sep 26, 2012 10:49:55 AM CDT> <Info> <Cluster> <Server3> <server3a> <[ACTIVE] ExecuteThread: '43' for queu
    e: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1348674595168> <BEA-000112> <Removing decs5a jvmid:16589531
    05634154558S:10.110.7.115:[8011,8011,-1,-1,-1,-1,-1]:decs:decs5a from cluster view due to timeout.>
    ####<Sep 26, 2012 10:49:59 AM CDT> <Info> <Cluster> <<Server3> <server3a> <weblogic.cluster.MessageReceiver> <<W
    LS Kernel>> <> <> <1348674599925> <BEA-000115> <Lost 2 multicast message(s).>
    ####<Sep 26, 2012 10:50:00 AM CDT> <Info> <Cluster><Server3> <server3a> <[ACTIVE] ExecuteThread: '16' for queu
    e: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1348674600471> <BEA-000111> <Adding decs5a with ID 16589531
    05634154558S:10.110.7.115:[8011,8011,-1,-1,-1,-1,-1]:server:server5a to cluster: decscluster view.>
    ####<Sep 26, 2012 10:50:05 AM CDT> <Info> <Cluster> <Server3> <server3a> <[ACTIVE] ExecuteThread: '36' for queu
    e: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1348674605176> <BEA-000128> <Updating 1658953105634154558S:
    10.110.7.115:[8011,8011,-1,-1,-1,-1,-1]:server:server5a in the cluster.>
    ####<Sep 26, 2012 10:50:24 AM CDT> <Info> <Cluster> <Server3> <server3a> <weblogic.cluster.MessageReceiver> <<W
    LS Kernel>> <> <> <1348674624426> <BEA-000115> <Lost 1 multicast message(s).>
    javax.ejb.EJBException
    ####<Sep 26, 2012 11:26:16 AM CDT> <Warning> <JNDI> <dal604se115.dynamex.com> <decs5a> <[ACTIVE] ExecuteThread: '12' for queu
    e: 'weblogic.kernel.Default (self-tuning)'> <kwplatt> <> <> <1348676776719> <BEA-050007> <An attempt was made to look up non-
    versioned global resource "" from an application version "administration.war [Version=3.20.1.1398]". This can potentially cau
    se conflict of the global resource usages among multiple application versions.>
    ####<Oct 3, 2012 2:27:13 PM CDT> <Error> <Cluster> <server1> <server1a> <[ACTIVE] ExecuteThread: '7' for queue:
    'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1349292433387> <BEA-000117> <Received a stale replication reques
    t for object 8042261441738434683.>
    ####<Oct 3, 2012 10:51:08 PM CDT> <Warning> <Cluster> <server1> <server1a> <[ACTIVE] ExecuteThread: '6' for queu
    e: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1349322668940> <BEA-000184> <This server did not attempt to
    renew its lease for 698 seconds. The server may restarted or migrated.>
    ####<Oct 3, 2012 10:51:08 PM CDT> <Info> <Cluster> <<server1> <server1a> <[ACTIVE] ExecuteThread: '6' for queue:
    'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1349322668949> <BEA-000185> <Despite the delay in the lease-ren
    ewing thread, the server has successfully renewed its lease. It is no longer in imminent risk of being migrated or restarted.>
    ####<Oct 3, 2012 10:40:23 PM CDT> <Info> <Server> <server3> <server3a> <[ACTIVE] ExecuteThread: '17' for queue:
    'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1349322023616> <BEA-002634> <The server "AdminServer" disconnec
    ted from this server.>
    ####<Oct 3, 2012 10:40:20 PM CDT> <Info> <Server> <server5> <server5a> <[STANDBY] ExecuteThread: '21' for queue:**
    'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1349322020161> <BEA-002634> <The server "AdminServer" disconne**
    cted from this server.>**
    Thanks in advance..!
    Imran khan.

    Most of these are around clustering. I would suggest you read the section on clustering in the documentation, it will explain most of this
    http://docs.oracle.com/cd/E11035_01/wls100/cluster/index.html
    For instance
    ####<Sep 26, 2012 10:49:59 AM CDT> <Info> <Cluster> <<Server3> <server3a> <weblogic.cluster.MessageReceiver> <<W
    LS Kernel>> <1348674599925> <BEA-000115> <Lost 2 multicast message(s).>
    ####<Sep 26, 2012 10:50:00 AM CDT> <Info> <Cluster><Server3> <server3a> <[ACTIVE] ExecuteThread: '16' for queu
    e: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <1348674600471> <BEA-000111> <Adding decs5a with ID 16589531
    05634154558S:10.110.7.115:[8011,8011,-1,-1,-1,-1,-1]:server:server5a to cluster: decscluster view.>
    is to do with communication between the servers in a cluster, using multicast, and how the server is losing (not receiving or not receiving a response for) multicast messages broadcast to the cluster.
    ####<Oct 3, 2012 2:27:13 PM CDT> <Error> <Cluster> <server1> <server1a> <[ACTIVE] ExecuteThread: '7' for queue:
    'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <1349292433387> <BEA-000117> <Received a stale replication reques
    t for object 8042261441738434683.>
    This is to do with session replication between the servers.
    ####<Oct 3, 2012 10:51:08 PM CDT> <Warning> <Cluster> <server1> <server1a> <[ACTIVE] ExecuteThread: '6' for queu
    e: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <1349322668940> <BEA-000184> <This server did not attempt to
    renew its lease for 698 seconds. The server may restarted or migrated.>
    This is leasing and how the server is not contacting the master manager server to maintain it's lease.
    ####<Oct 3, 2012 10:51:08 PM CDT> <Info> <Cluster> <<server1> <server1a> <[ACTIVE] ExecuteThread: '6' for queue:
    'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <1349322668949> <BEA-000185> <Despite the delay in the lease-ren
    ewing thread, the server has successfully renewed its lease. It is no longer in imminent risk of being migrated or restarted.>
    as is this, only here the lease is taken.
    ####<Oct 3, 2012 10:40:20 PM CDT> <Info> <Server> <server5> <server5a> <[STANDBY] ExecuteThread: '21' for queue:**
    'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <1349322020161> <BEA-002634> <The server "AdminServer" disconne**
    cted from this server.>**
    this means that the managed server can no reach the admin server, probably as it's been restarted.

  • Cluster replication issue in 12c

    Hi all,
    After the installation of Weblogic Server 12c we have got cluster replication problems in our application test environment.
    Although the same configuration used to work with Weblogic Server 10.3 now the secondary server seems to be removed and added to the cluster randomly.
    +<[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1332511394471> <BEA-000113> <Removing WL12c_ManagedServer_2 jvmid:4384922070744243395S:172.22.250.219:[10002,10002,-1,-1,-1,-1,-1]:project_domain:WL12c_ManagedServer_2 from cluster view due to PeerGone.>+
    +<WL12c_ManagedServer_1> <weblogic.cluster.MessageReceiver> <<WLS Kernel>> <> <> <1332511403456> <BEA-003107> <Lost 2 unicast message(s).>+
    +<WL12c_ManagedServer_1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1332511403457> <BEA-002936> <maximum thread constraint weblogic.unicast.ForwardingWorkManager is reached>+
    +<WL12c_ManagedServer_1> <[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1332511403460> <BEA-000111> <Adding WL12c_ManagedServer_2 with ID 4384922070744243395S:172.22.250.219:[10002,10002,-1,-1,-1,-1,-1]:project_domain:WL12c_ManagedServer_2 to cluster: ClusterWL12c view.>+
    +<WL12c_ManagedServer_1> <[STANDBY] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1332511403462> <BEA-000128> <Updating 4384922070744243395S:172.22.250.219:[10002,10002,-1,-1,-1,-1,-1]:project_domain:WL12c_ManagedServer_2 in the cluster.>+
    +<WL12c_ManagedServer_1> <ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'> <<WLS Kernel>> <> <> <1332511413393> <BEA-002936> <maximum thread constraint weblogic.unicast.DispatchWorkManager is reached>+
    Further I see in the HttpClusterServlet log file that the secondary server hash is occasionally set to *!NONE!*
    Environment Details (all servers are running on one physical server):
    -     1 Node Server
    -     1 Cluster (unicast) with 2 managed servers
    -     1 Standalone managed server used as proxy (load balancer) based on HttpClusterServlet.
    -     Linux redhat 2.6.18-128
    -     Applications have been compiled using the weblogic server JDK
    Part of config.xml:
    +<server>+
    +<name>AdminServer</name>+
    +<listen-port>17001</listen-port>+
    +<listen-address></listen-address>+
    +</server>+
    +<server>+
    +<name>WL12c_ManagedServer_1</name>+
    +<ssl>+
    +<enabled>false</enabled>+
    +</ssl>+
    +<machine>WL12c_Machine</machine>+
    +<listen-port>10001</listen-port>+
    +<cluster>ClusterWL12c</cluster>+
    +<listen-address></listen-address>+
    +<server-start>+
    +<arguments>-Djava.awt.headless=true</arguments>+
    +</server-start>+
    +<jta-migratable-target>+
    +<name>WL12c_ManagedServer_1</name>+
    +<user-preferred-server>WL12c_ManagedServer_1</user-preferred-server>+
    +<cluster>ClusterWL12c</cluster>+
    +</jta-migratable-target>+
    +<weblogic-plugin-enabled>false</weblogic-plugin-enabled>+
    +</server>+
    +<server>+
    +<name>WL12c_ManagedServer_2</name>+
    +<ssl>+
    +<enabled>false</enabled>+
    +</ssl>+
    +<machine>WL12c_Machine</machine>+
    +<listen-port>10002</listen-port>+
    +<cluster>ClusterWL12c</cluster>+
    +<listen-address></listen-address>+
    +<server-start>+
    +<arguments>-Djava.awt.headless=true</arguments>+
    +</server-start>+
    +<jta-migratable-target>+
    +<name>WL12c_ManagedServer_2</name>+
    +<user-preferred-server>WL12c_ManagedServer_2</user-preferred-server>+
    +<cluster>ClusterWL12c</cluster>+
    +</jta-migratable-target>+
    +<weblogic-plugin-enabled>false</weblogic-plugin-enabled>+
    +</server>+
    +<server>+
    +<name>WL12c_ProxyServer</name>+
    +<ssl>+
    +<enabled>false</enabled>+
    +</ssl>+
    +<machine>WL12c_Machine</machine>+
    +<listen-port>9001</listen-port>+
    +<cluster xsi:nil="true"></cluster>+
    +<web-server>+
    +<web-server-log>+
    +<number-of-files-limited>false</number-of-files-limited>+
    +</web-server-log>+
    +</web-server>+
    +<listen-address></listen-address>+
    +</server>+
    +<cluster>+
    +<name>ClusterWL12c</name>+
    +<cluster-messaging-mode>unicast</cluster-messaging-mode>+
    +<weblogic-plugin-enabled>false</weblogic-plugin-enabled>+
    +<frontend-host>build-env.rtd</frontend-host>+
    +<frontend-http-port>9001</frontend-http-port>+
    +<number-of-servers-in-cluster-address>2</number-of-servers-in-cluster-address>+
    +</cluster>+
    Does anyone have an idea how to solve this issue?

    Seeing the same issue with WebLogic 12c on AIX 7.1, with IBM JDK SR3. 
    The same basic config works correctly on Linux and Windows.
    Same as OP, we are using unicast cluster messaging with two managed servers on the same OS instance.   We use Apache for the front-end proxy.  However, the nodes are dropped and added from the cluster whenever they attempt to send a cluster message of any kind.   Although they are quickly re-added, application errors result when performing any operation on a node while it is excluded from the cluster.

Maybe you are looking for