Session replication failed
I have multi server environment. Two phisical servers.
Balanced by a hardware load balancer. Each of the server have
multiple instances of ColdFusion. Main instance (cfusion), and
cf_instance_server1_id_1 and cf_instance_server2_2.
CFUSION instance is used as administrative instance only,
other two instances are hosting independent applications.
cf_instance_server1_id_1 is clustered with
cf_instance_server2_id_1, same applies for second instance.
jrun/logs folder is filling up fast with following error
jsession is on.
Please advise
Mikkhait wrote:
> CFUSION instance is used as administrative instance
only, other two instances
> are hosting independent applications.
cf_instance_server1_id_1 is clustered
> with cf_instance_server2_id_1, same applies for second
instance.
> 11/07 10:16:39 error Setup of session replication
failed.
> [1]java.net.ConnectException: Connection refused:
connect
You have a network issue. Make sure there are no firewalls
between both
servers and your routing is set up correctly. If that doesn't
work, we
need a dump of the route table, the interface configuration
and the
overview of instances from the cluster manager.
Jochem
Jochem van Dieten
Adobe Community Expert for ColdFusion
Similar Messages
-
Http session replication fails in cluster
Hello everybody.
I have some problems with HTTP session replication in WebLlogic cluster environment. I have a cluster with 2 nodes and application deployed there. Application is configured with:
<session-param>
<param-name>PersistentStoreType</param-name>
<param-value>replicated</param-value>
</session-param>
in weblogic.xml
WebLogic plug-in for Apache webserver is configured properly as described in documentation.
But when I try to make experiment to enter the application, make some activities, look in console where I was redirected by apache proxy and manually shut down the node where request was sent, I loose my HTTP session with all data there (all the beans stored in session are Serializable). Replication doesn't work correctly. May be I've missed something in configuration? How can I configure my application to provide correctness session replication?
Thanks for advice.
ThanksThanks for response!
I'm using WebLogic 8.1 SP 4 and configured Apache proxy as described in documentation for load balancing. The only section I have in httpd.conf is next:
<Location /HTTPClnt>
SetHandler weblogic-handler
</Location>
<IfModule mod_weblogic.c>
WebLogicCluster serv1:7541,serv2:7541
MatchExpression *.*
Debug ON
WLLogFile /www/tmp/global_proxy.log
WLTempDir "/www/tmp"
DebugConfigInfo On
KeepAliveEnabled ON
KeepAliveSecs 15
</IfModule>
<Location /myApp>
SetHandler weblogic-handler
WebLogicCluster serv1:7541,serv2:7541
</Location>and the string to include weblogic proxy module for apache:
LoadModule weblogic_module modules/mod_wl_20.soI've configured CookiesEnabled=true in weblogic.xml, but it didn't help. About session specification in httpd.conf - where can I read about that? I've just configured apache according to manual from here:
[url http://e-docs.bea.com/wls/docs92/plugins/apache.html]http://e-docs.bea.com/wls/docs92/plugins/apache.html
Thanks -
hi all,
we have two weblogic servers 6.1 in a cluster (running under windows NT) proxied
by a apache webserver 1.3.12 with the bea plugin. (running under SUN solaris).
We try to test the failover behavier of the cluster and the session in-memory-replication
in particular.
We have implemented a little servlet which is just printing the session id on
the server console and stores some data in the session context. When we send multiple
request from a single client to this servlet, the proxy connects to the specific
server in which the session was originally established and the original session
id and the data in the session context keeps identical. But when we kill this
server (perssing CTRL+C) and the proxy connects to the other server in the cluster,
a new session-id is created and the session data is lost. It seems that the session
is not replicated.
May be this is a configuration problem. Our http.conf files includes the following
plugin-configuration:
AddModule mod_so.c
LoadModule weblogic_module libexec/mod_wl_ssl.so
AddModule mod_weblogic.c
<IfModule mod_weblogic.c>
WebLogicCluster ws110009:7001,ws110344:7001
MatchExpression *.jsp
Debug ALL
DebugConfigInfo ON
ConnectTimeoutSecs 3
ConnectRetrySecs 1
HungServerRecoverSecs 30
</IfModule>
<Location /hallo>
SetHandler weblogic-handler
</Location>
Our test servlet is very simple:
import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;
public class HelloWorld extends HttpServlet
public void service(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, java.io.IOException
HttpSession session;
String strAttribute;
String strSessionAttribute = req.getRemoteHost();
PrintWriter out = resp.getWriter();
session = req.getSession(true);
resp.setContentType("text/html");
System.out.println("*************** im Servlet ***************");
System.out.println("Session "+session);
if(session != null)
System.out.println("SessionID "+session.getId());
strAttribute = (String)session.getAttribute("TEST");
System.out.println("Attribute aus Session "+strAttribute);
System.out.println("Attribute in Session gesetzt: "+strSessionAttribute);
if(strSessionAttribute != null)
session.setAttribute("TEST", strSessionAttribute);
}else
session.setAttribute("TEST", "war nix");
out.println("<html>");
out.println("<head><title>Hello World</title></head>");
out.println("<body>");
out.println("<h1>Hello World</h1>");
out.println("<a href='http://sunrise:8080/hallo'><h1>Klick hier</h1></a>");
out.println("</body></html>");
System.out.println("*************** ******* ***************");
and we use the following weblogic.xml for deployment:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE weblogic-web-app PUBLIC "-//BEA Systems, Inc.//DTD Web Application 6.0//EN"
"http://www.bea.com/servers/wls600/dtd/weblogic-web-jar.dtd">
<weblogic-web-app>
<session-descriptor>
<session-param>
<param-name>
PersistentStoreType
</param-name>
<param-value>
replicated
</param-value>
</session-param>
<session-param>
<param-name>
URLRewritingEnabled
</param-name>
<param-value>
false
</param-value>
</session-param>
</session-descriptor>
</weblogic-web-app>
We call the servlet three times and the primary server console looks like this:
*************** im Servlet ***************
Session weblogic.servlet.internal.session.ReplicatedSessionData@2746
SessionID O4O2WnsI55wDt3aUkfxjYwTAWs2opiTCl4PAPjun4EMu2HaKB2T4!-8932999811214751
489!167772240!7001!7002!NONE!998489690052
Attribute aus Session null
Attribute in Session gesetzt: ws110341.pass-consulting.com
*************** im Servlet ***************
Session weblogic.servlet.internal.session.ReplicatedSessionData@2746
SessionID O4O2WnsI55wDt3aUkfxjYwTAWs2opiTCl4PAPjun4EMu2HaKB2T4!-8932999811214751
489!167772240!7001!7002!NONE!998489690052
Attribute aus Session ws110341.pass-consulting.com
Attribute in Session gesetzt: ws110341.pass-consulting.com
*************** im Servlet ***************
Session weblogic.servlet.internal.session.ReplicatedSessionData@2746
SessionID O4O2WnsI55wDt3aUkfxjYwTAWs2opiTCl4PAPjun4EMu2HaKB2T4!-8932999811214751
489!167772240!7001!7002!NONE!998489690052
Attribute aus Session ws110341.pass-consulting.com
Attribute in Session gesetzt: ws110341.pass-consulting.com
When we kill this server and the request goes to the second server the console
looks like this:
*************** im Servlet ***************
Session weblogic.servlet.internal.session.ReplicatedSessionData@4c678a
SessionID O4O2WnsI55wDt3aUkfxjYwTAWs2opiTCl4PAPjun4EMu2HaKB2T4!-1303983227923606
289!167772222!7001!7002!NONE!998489284083
Attribute aus Session null
Attribute in Session gesetzt: ws110341.pass-consulting.com
*************** im Servlet ***************
Session weblogic.servlet.internal.session.ReplicatedSessionData@4c678a
SessionID O4O2WnsI55wDt3aUkfxjYwTAWs2opiTCl4PAPjun4EMu2HaKB2T4!-1303983227923606
289!167772222!7001!7002!NONE!998489284083
Attribute aus Session ws110341.pass-consulting.com
Attribute in Session gesetzt: ws110341.pass-consulting.com
The session id has changed and the session-attribute is lost.
Have anyone an idea ?
Thanks
Frank
Make sure clustering is properly setup. Looks into the log file for any errors.
Check the plugin param docs DebugConfigInfo to see the dynamic cluster list,
which gives an indication of wether clustering is setup properly or not.
session.getClass() will give you idea which replication type is the server using
for the sessions.
--Vinod.
"Frank Möller" wrote:
> hi all,
>
> we have two weblogic servers 6.1 in a cluster (running under windows NT) proxied
> by a apache webserver 1.3.12 with the bea plugin. (running under SUN solaris).
> We try to test the failover behavier of the cluster and the session in-memory-replication
> in particular.
>
> We have implemented a little servlet which is just printing the session id on
> the server console and stores some data in the session context. When we send multiple
> request from a single client to this servlet, the proxy connects to the specific
> server in which the session was originally established and the original session
> id and the data in the session context keeps identical. But when we kill this
> server (perssing CTRL+C) and the proxy connects to the other server in the cluster,
> a new session-id is created and the session data is lost. It seems that the session
> is not replicated.
>
> May be this is a configuration problem. Our http.conf files includes the following
> plugin-configuration:
>
> AddModule mod_so.c
>
> LoadModule weblogic_module libexec/mod_wl_ssl.so
> AddModule mod_weblogic.c
>
> <IfModule mod_weblogic.c>
> WebLogicCluster ws110009:7001,ws110344:7001
> MatchExpression *.jsp
> Debug ALL
> DebugConfigInfo ON
> ConnectTimeoutSecs 3
> ConnectRetrySecs 1
> HungServerRecoverSecs 30
> </IfModule>
>
> <Location /hallo>
> SetHandler weblogic-handler
> </Location>
>
> Our test servlet is very simple:
>
> import java.io.*;
> import javax.servlet.*;
> import javax.servlet.http.*;
>
> public class HelloWorld extends HttpServlet
> {
>
> public void service(HttpServletRequest req, HttpServletResponse resp)
> throws ServletException, java.io.IOException
> {
>
> HttpSession session;
> String strAttribute;
> String strSessionAttribute = req.getRemoteHost();
>
>
> PrintWriter out = resp.getWriter();
> session = req.getSession(true);
>
>
> resp.setContentType("text/html");
>
> System.out.println("*************** im Servlet ***************");
> System.out.println("Session "+session);
> if(session != null)
> {
> System.out.println("SessionID "+session.getId());
>
> strAttribute = (String)session.getAttribute("TEST");
> System.out.println("Attribute aus Session "+strAttribute);
> System.out.println("Attribute in Session gesetzt: "+strSessionAttribute);
>
> if(strSessionAttribute != null)
> {
> session.setAttribute("TEST", strSessionAttribute);
> }else
> {
> session.setAttribute("TEST", "war nix");
> }
> }
> out.println("<html>");
> out.println("<head><title>Hello World</title></head>");
> out.println("<body>");
> out.println("<h1>Hello World</h1>");
> out.println("<a href='http://sunrise:8080/hallo'><h1>Klick hier</h1></a>");
> out.println("</body></html>");
> System.out.println("*************** ******* ***************");
>
> }
> }
>
> and we use the following weblogic.xml for deployment:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE weblogic-web-app PUBLIC "-//BEA Systems, Inc.//DTD Web Application 6.0//EN"
> "http://www.bea.com/servers/wls600/dtd/weblogic-web-jar.dtd">
> <weblogic-web-app>
> <session-descriptor>
> <session-param>
> <param-name>
> PersistentStoreType
> </param-name>
> <param-value>
> replicated
> </param-value>
> </session-param>
> <session-param>
> <param-name>
> URLRewritingEnabled
> </param-name>
> <param-value>
> false
> </param-value>
> </session-param>
> </session-descriptor>
> </weblogic-web-app>
>
> We call the servlet three times and the primary server console looks like this:
>
> *************** im Servlet ***************
> Session weblogic.servlet.internal.session.ReplicatedSessionData@2746
> SessionID O4O2WnsI55wDt3aUkfxjYwTAWs2opiTCl4PAPjun4EMu2HaKB2T4!-8932999811214751
> 489!167772240!7001!7002!NONE!998489690052
> Attribute aus Session null
> Attribute in Session gesetzt: ws110341.pass-consulting.com
> *************** ******* ***************
> *************** im Servlet ***************
> Session weblogic.servlet.internal.session.ReplicatedSessionData@2746
> SessionID O4O2WnsI55wDt3aUkfxjYwTAWs2opiTCl4PAPjun4EMu2HaKB2T4!-8932999811214751
> 489!167772240!7001!7002!NONE!998489690052
> Attribute aus Session ws110341.pass-consulting.com
> Attribute in Session gesetzt: ws110341.pass-consulting.com
> *************** ******* ***************
> *************** im Servlet ***************
> Session weblogic.servlet.internal.session.ReplicatedSessionData@2746
> SessionID O4O2WnsI55wDt3aUkfxjYwTAWs2opiTCl4PAPjun4EMu2HaKB2T4!-8932999811214751
> 489!167772240!7001!7002!NONE!998489690052
> Attribute aus Session ws110341.pass-consulting.com
> Attribute in Session gesetzt: ws110341.pass-consulting.com
> *************** ******* ***************
>
> When we kill this server and the request goes to the second server the console
> looks like this:
>
> *************** im Servlet ***************
> Session weblogic.servlet.internal.session.ReplicatedSessionData@4c678a
> SessionID O4O2WnsI55wDt3aUkfxjYwTAWs2opiTCl4PAPjun4EMu2HaKB2T4!-1303983227923606
> 289!167772222!7001!7002!NONE!998489284083
> Attribute aus Session null
> Attribute in Session gesetzt: ws110341.pass-consulting.com
> *************** ******* ***************
> *************** im Servlet ***************
> Session weblogic.servlet.internal.session.ReplicatedSessionData@4c678a
> SessionID O4O2WnsI55wDt3aUkfxjYwTAWs2opiTCl4PAPjun4EMu2HaKB2T4!-1303983227923606
> 289!167772222!7001!7002!NONE!998489284083
> Attribute aus Session ws110341.pass-consulting.com
> Attribute in Session gesetzt: ws110341.pass-consulting.com
> *************** ******* ***************
>
> The session id has changed and the session-attribute is lost.
>
> Have anyone an idea ?
>
> Thanks
> Frank
-
Hi,
CFMX 7.01 MULTISERVER:
I am facing a problem, session are not replicating on two CF
instances on same server. Below is Jrun log file error detail
30/05 17:18:10 error Setup of session replication failed.
[1]java.rmi.RemoteException: The web application
'cfusion.ear#cfusion.war' could not be found to accept sessions for
replication.
at
jrun.servlet.session.SessionReplicationService.replicate(SessionReplicationService.java:8 0)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown
Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
at sun.rmi.transport.Transport$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native
Method)
at sun.rmi.transport.Transport.serviceCall(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown
Source)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
CFMX 7.01 MULTISERVER:
My set-up, installed as multiserver. create new instance from
CF Admin. Cluster them from Jrun Admin. I followed almost all
instructions from live doc, and also help from other blogs.
So far i could not findout why session can not replicate on
same machine.
Is any body have idea about above error.
Thanks for your help in-advanceIn my config.xml, I have:
<cluster>
<name>MyCluster</name>
<cluster-address/>
<default-load-algorithm>round-robin</default-load-algorithm>
<cluster-messaging-mode>unicast</cluster-messaging-mode>
<frontend-host>192.168.6.6</frontend-host>
<frontend-http-port>80</frontend-http-port>
<frontend-https-port>443</frontend-https-port>
</cluster>
I already posted my weblogic.xml -
CF8 +session replication +stackoverflow
Good day
I'm running a CentOS cluster releas 5.3 64bit version. There are two servers in the cluster, each running 2 CF8.1 servers in a cluster mode.
I have a dedicated jvm config file for each server, so that I can set the mem specs per server. All 4 are running on 1Gig memory for the JVM.
The admin server process on each server is running with max 500M JVM space.
I'm "using weighted round robin" with "sticky sessions" and "replicate sessions" selected.
I have now started the jconsole and monitored one of the servers. With only 300Meg alocated in the JVM, I have this error.
11/09 08:06:05 Information [jrpp-4112] - Called include
11/09 08:06:05 error Setup of session replication failed.
java.lang.StackOverflowError
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1161)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
It is not effecting the rest of the applicatin (so it seems and nobody has complained yet), bit is not acceptable as it raises the alarm every time.
These are the CF parameters
Version
8,0,1,195765
Edition
Enterprise
Serial Number
Operating System
UNIX
OS Version
2.6.18-128.4.1.el5
JVM Details
Java Version
1.6.0_04
Java Vendor
Sun Microsystems Inc.
Java Vendor URL
http://java.sun.com/
Anybody with a solution/idea?
Tks
Andre
Message was edited by Jochem van Dieten to remive a serial numberOops - doing so many things simultanious at the time that I missed the serial no. Tks for editing.
I will go and have a serious look at the session variable usage.
The application is a restricted-access type with the majority of the the application (80%) running in the background.
The application uses session variables quite heavily for authentication and other access control mechanisms.
Maybee it is a better option to just disable the session replication. If a server fails, the user must simply re-authorize him/herself.
Tks
Andre -
Unstable session replication in a HA cluster (CF10)
Hi,
We have tried to create a HA cluster with requests being distributed round robin to N instances of coldfusion, we are NOT using sticky sessions as we are replication session state to all cf instances. What we are seing is that all is fine with low to moderate load, however under heavy load and at random times the replication fails and leads to things in session scope not working. This manifests in users not being able to login to our application (we store a token in session scope to store logged in status).
Again key point, under low to moderate load it all works fine, users are directed to random nodes in the cluster and their session is picked up fine as the session is distributed to all nodes,so pretty confident config is right.
Linux servers using CF10 with update 12 applied. Also running is fusion reactor 5.04 on all instances. Each instance has a 64GB heap, Java 7.0.15 (latest certified).
Firstly apache setup.
workers.properties
worker.list=balancer, jkstatus
worker.jkstatus.type=status
worker.balancer.type=lb
worker.balancer.balance_workers=cfusion_master,cfusion_slave2,cfusion_slave1
worker.balancer.method=R
worker.balancer.sticky_session=False
worker.balancer.ping_mode=A
worker.cfusion_master.type=ajp13
worker.cfusion_master.host=localhost
worker.cfusion_master.port=8012
worker.cfusion_master.max_reuse_connections=250
worker.cfusion_master.lbfactor=100
worker.cfusion_slave2.reference=worker.cfusion_master
worker.cfusion_slave2.port=8014
worker.cfusion_slave1.reference=worker.cfusion_master
worker.cfusion_slave1.port=8013
Now the server.xml from 2 nodes (as an example if I run a 2 node cluster)
One of the configs from a server in the cluster
<Server port="8007" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on">
</Listener>
<Listener className="org.apache.catalina.core.JasperListener">
</Listener>
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener">
</Listener>
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener">
</Listener>
<GlobalNamingResources>
<Resource description="User database that can be updated and saved" name="UserDatabase" pathname="conf/tomcat-users.xml" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" type="org.apache.catalina.UserDatabase" auth="Container">
</Resource>
</GlobalNamingResources>
<Service name="Catalina">
<Executor name="tomcatThreadPool" minSpareThreads="4" maxThreads="150" namePrefix="catalina-exec-">
</Executor>
<Connector port="8012" protocol="AJP/1.3" connectionTimeout="600000" redirectPort="8445" tomcatAuthentication="false">
</Connector>
<Engine jvmRoute="cfusion" name="Catalina" defaultHost="localhost">
<Realm className="org.apache.catalina.realm.LockOutRealm">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase">
</Realm>
</Realm>
<Host name="localhost" autoDeploy="false" unpackWARs="true" appBase="webapps">
<Valve pattern="%h %l %u %t "%r" %s %b" directory="logs" prefix="localhost_access_log." className="org.apache.catalina.valves.AccessLogValve" suffix=".txt" resolveHosts="false">
</Valve>
</Host>
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8">
<Manager notifyListenersOnReplication="true" expireSessionsOnShutdown="false" className="org.apache.catalina.ha.session.DeltaManager">
</Manager>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership port="45564" dropTime="3000" address="228.0.0.4" className="org.apache.catalina.tribes.membership.McastService" frequency="500">
</Membership>
<Receiver port="4001" autoBind="100" address="auto" selectorTimeout="5000" maxThreads="6" className="org.apache.catalina.tribes.transport.nio.NioReceiver">
</Receiver>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender">
</Transport>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector">
</Interceptor>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor">
</Interceptor>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter="">
</Valve>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve">
</Valve>
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener">
</ClusterListener>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener">
</ClusterListener>
</Cluster>
</Engine>
<Connector port="8499" protocol="org.apache.coyote.http11.Http11NioProtocol" connectionTimeout="20000" redirectPort="8443" executor="tomcatThreadPool">
</Connector>
</Service>
</Server>
Config from one of the other nodes
<Server port="8008" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on">
</Listener>
<Listener className="org.apache.catalina.core.JasperListener">
</Listener>
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener">
</Listener>
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener">
</Listener>
<GlobalNamingResources>
<Resource description="User database that can be updated and saved" name="UserDatabase" pathname="conf/tomcat-users.xml" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" type="org.apache.catalina.UserDatabase" auth="Container">
</Resource>
</GlobalNamingResources>
<Service name="Catalina">
<Executor name="tomcatThreadPool" minSpareThreads="4" maxThreads="150" namePrefix="catalina-exec-">
</Executor>
<Connector port="8013" protocol="AJP/1.3" connectionTimeout="600000" redirectPort="8446" tomcatAuthentication="false">
</Connector>
<Engine jvmRoute="cfusion" name="Catalina" defaultHost="localhost">
<Realm className="org.apache.catalina.realm.LockOutRealm">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase">
</Realm>
</Realm>
<Host name="localhost" autoDeploy="false" unpackWARs="true" appBase="webapps">
<Valve pattern="%h %l %u %t "%r" %s %b" directory="logs" prefix="localhost_access_log." className="org.apache.catalina.valves.AccessLogValve" suffix=".txt" resolveHosts="false">
</Valve>
</Host>
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8">
<Manager notifyListenersOnReplication="true" expireSessionsOnShutdown="false" className="org.apache.catalina.ha.session.DeltaManager">
</Manager>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership port="45564" dropTime="3000" address="228.0.0.4" className="org.apache.catalina.tribes.membership.McastService" frequency="500">
</Membership>
<Receiver port="4002" autoBind="100" address="auto" selectorTimeout="5000" maxThreads="6" className="org.apache.catalina.tribes.transport.nio.NioReceiver">
</Receiver>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender">
</Transport>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector">
</Interceptor>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor">
</Interceptor>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter="">
</Valve>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve">
</Valve>
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener">
</ClusterListener>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener">
</ClusterListener>
</Cluster>
</Engine>
<Connector port="8500" protocol="org.apache.coyote.http11.Http11NioProtocol" connectionTimeout="20000" redirectPort="8443" executor="tomcatThreadPool">
</Connector>
</Service>
</Server>
So what do i see in the logs?. Well sometimes I see exceptions like this
Mar 05, 2014 9:55:19 PM org.apache.catalina.ha.session.DeltaManager messageReceived
SEVERE: Manager [localhost#/]: Unable to receive message through TCP channel
java.lang.IllegalStateException: removeAttribute: Session already invalidated
at org.apache.catalina.ha.session.DeltaSession.removeAttribute(DeltaSession.java:617)
at org.apache.catalina.ha.session.DeltaRequest.execute(DeltaRequest.java:171)
at org.apache.catalina.ha.session.DeltaManager.handleSESSION_DELTA(DeltaManager.java:1347)
at org.apache.catalina.ha.session.DeltaManager.messageReceived(DeltaManager.java:1293)
at org.apache.catalina.ha.session.DeltaManager.messageDataReceived(DeltaManager.java:1014)
at org.apache.catalina.ha.session.ClusterSessionListener.messageReceived(ClusterSessionListe ner.java:92)
at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:897)
at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:878)
at org.apache.catalina.tribes.group.GroupChannel.messageReceived(GroupChannel.java:278)
at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelIntercepto rBase.java:84)
at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.messageReceived(TcpFailu reDetector.java:113)
at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelIntercepto rBase.java:84)
at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelIntercepto rBase.java:84)
at org.apache.catalina.tribes.group.ChannelCoordinator.messageReceived(ChannelCoordinator.ja va:253)
at org.apache.catalina.tribes.transport.ReceiverBase.messageDataReceived(ReceiverBase.java:2 87)
at org.apache.catalina.tribes.transport.nio.NioReplicationTask.drainChannel(NioReplicationTa sk.java:212)
at org.apache.catalina.tribes.transport.nio.NioReplicationTask.run(NioReplicationTask.java:1 01)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
I'm unsure why this happens as tribes uses certified mesaging so it should have resent right?, in any case I believe I can change it so messages are not sent asynchronously, should sort this out.
I see (good) messages like this
Mar 05, 2014 9:42:19 PM org.apache.catalina.ha.session.DeltaManager startInternal
INFO: Register manager localhost#/ to cluster element Engine with name Catalina
Mar 05, 2014 9:42:19 PM org.apache.catalina.ha.session.DeltaManager startInternal
INFO: Starting clustering manager at localhost#/
Mar 05, 2014 9:42:19 PM org.apache.catalina.ha.session.DeltaManager getAllClusterSessions
INFO: Manager [localhost#/], requesting session state from org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 128, 50}:4001,{192, 168, 128, 50},4001, alive=68824148, securePort=-1, UDP Port=-1, id={123 126 89 39 96 -59 69 8 -113 79 51 122 25 108 -11 -110 }, payload={}, command={}, domain={}, ]. This operation will timeout if no session state has been received within 60 seconds.
Mar 05, 2014 9:42:20 PM org.apache.catalina.ha.session.DeltaManager waitForSendAllSessions
INFO: Manager [localhost#/]; session state send at 3/5/14 9:42 PM received in 929 ms.
Mar 05, 2014 9:42:20 PM org.apache.catalina.ha.session.JvmRouteBinderValve startInternal
INFO: JvmRouteBinderValve started
So session state dies appear to be flying around the cluster, I do nightly restarts of some of the nodes due to another issue I have with an ever growing heap (separate issue), interestingly I also see nodes leave and join the cluster, again this is good (shows the multicast is working, and also that replication should be working).
Mar 05, 2014 2:30:16 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared
INFO: Verification complete. Member disappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 128, 50}:4001,{192, 168, 128, 50},4001, alive=18629101, securePort=-1, UDP Port=-1, id={-2 65 10 -79 53 -75 76 52 -99 63 -90 -120 34 -89 -14 100 }, payload={}, command={66 65 66 89 45 65 76 69 88 ...(9)}, domain={}, ]]
Mar 05, 2014 2:30:16 AM org.apache.catalina.ha.tcp.SimpleTcpCluster memberDisappeared
INFO: Received member disappeared:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 128, 50}:4001,{192, 168, 128, 50},4001, alive=18629101, securePort=-1, UDP Port=-1, id={-2 65 10 -79 53 -75 76 52 -99 63 -90 -120 34 -89 -14 100 }, payload={}, command={66 65 66 89 45 65 76 69 88 ...(9)}, domain={}, ]
Mar 05, 2014 2:35:16 AM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 128, 50}:4001,{192, 168, 128, 50},4001, alive=1083, securePort=-1, UDP Port=-1, id={123 126 89 39 96 -59 69 8 -113 79 51 122 25 108 -11 -110 }, payload={}, command={}, domain={}, ]
So stuck now on how to proceed, to establish why at random times the replication fails, leading to cluster collapse. Could it be the size of the session?, I have a few CFCs stuffed into session scope, but perhaps when the load is high there is too many?. Things fail even with a cluster of 2 on one server, initially I had a 8 node cluster on 2 separate machines but when it failed it rolled it back to a cluster of 2 instances on the one server to see if that was stable (its not 100% which is what I need).
Any advice, points gratefully received.So Lynux, that’s an interesting sounding solution. Would be great if it made some difference for Guitsboy. We’ll see. I notice that you say you’ve not yet tried it, though, and fair enough. Thanks for offering it.
But I’m curious: did you ever resolve your original problem? And if not, hopefully you saw the note I just wrote to Guitsboy, asking him something that may well interest you if you still have your problem. On rereading this thread, from back in April, I’ve also had some new thoughts come to mind which I’ll share, if it may help either of you, or others with this seeming same issue.
To remind readers who may not want to review the whole thread, you had said originally that “all is fine with low to moderate load, however under heavy load and at random times the replication fails“, and that this failure “manifests in users not being able to login to our application (we store a token in session scope to store logged in status)”. Then it seems you may have concluded that things were down to the error you were seeing in the logs:
Mar 05, 2014 9:55:19 PM org.apache.catalina.ha.session.DeltaManager messageReceived
SEVERE: Manager : Unable to receive message through TCP channel
java.lang.IllegalStateException: removeAttribute: Session already invalidated
And now guitsboy reports seeing the same error.
But here’s the thing that came to mind for me tonight as I read this: you know, there can be a lot of other reasons that users can feel that they “lose their session”, even without using clustering and replication.
There are issues related sometimes to folks having duplicate session tokens (which can happen for various reasons, including perhaps ones in your code, and maybe only when people visit pages in a certain pattern, so that it happens only occasionally and not always).
Then there is an issue that can arise if you are supporting both http and https requests, where Tomcat (not CF) balks at that (see http://www.petefreitag.com/item/817.cfm, and though he shows a solution in IIS you should be able to implement a similar one in mod_rewrite if that was indeed perhaps your issue).
So I’d be curious if either of you may be in a position to have a failing client use any sort of client tool (like Chrome’s dev tools, or Firebug or Firefox’s new builtin tools, or IE’s f12 dev tools) to watch the communication between the client and the server, and especially to watch the cookies being sent. You guys both mention using jsessionid. Are they the same cookie value on each request? And/or are there more than jsessionid? I’ve seen it happen. There could be differences in the domain property reported for the cookie, the httponly property, the secure property, and so on. And you really do want to view the value sent from the client to the server, because if you view the cookie scope on the server a) it may show values set ON the server rather than sent TO the server, and b) it won’t show these additional cookie properties that were in play on the client. CF only sees the cookie name and value.
I’ve helped many people find out that this was the reason for the seeming session loss (and sometimes it was not all requests by all clients but perhaps only some requests for some clients, all on the same server). At least if this is the crux of the problem, you can then tackle WHY it’s happening. There can be many reasons, from code to configuration, so I won’t belabor them now.
But if either of you may be able to confirm this, perhaps we can help you both get a little closer to a real explanation and solution for your problem. Again, I’m just guessing a bit based on what you’ve written. I realize it may be that none of this is the problem and you have hit some other real unrelated bug. But I really feel confident that you ought to try to check this out first, as it’s indeed been the crux of problems for others, without respect to clustering. It seems worth ruling out, so that you don’t get misled chasing the problem on the assumption that it is about clustering.
As always, hope that helps.
/charlie -
Weblogic 7.0 sp1 cluster - session replication problem
Hi,
I have installed Weblogic 7.0 sp2 on Win NT. To test clustering
feature, I have installed one admin server and added two managed
servers. All are running on same box. I could deploy web application
to the cluster. Connection pools and every other resource is working
well with the cluster. However I couldn't get session replication to
work. I have modified web app descriptor, and set 'persistent store
type' to "replicated".
I accessed application from one managed server, in the middle of
session I modified the port number in the URL to point to other
managed server. It looks like second managed server has no idea of
that session, my app fails because of this.
Could you please help me out in this, Do I need to do any thing in
addition to the above. I couldn't find much in the BEA manual..
Thanks
Rao
For Web application like servlets/JSP, it is better to put one web server as proxy
plugin before your two managed servers and access your application through web
proxy. (You need set session as in-memory replicated either in weblogic.xml or
by console editor). Otherwise, you need record the session cookie from the first
serevr and send the cookie to the second server (not sure if it works). To access
EJB/JMS, use cluster URL like t3://server1:port1,server2:port2.
[email protected] (Rao) wrote:
>Hi,
>
>I have installed Weblogic 7.0 sp2 on Win NT. To test clustering
>feature, I have installed one admin server and added two managed
>servers. All are running on same box. I could deploy web application
>to the cluster. Connection pools and every other resource is working
>well with the cluster. However I couldn't get session replication to
>work. I have modified web app descriptor, and set 'persistent store
>type' to "replicated".
>
>I accessed application from one managed server, in the middle of
>session I modified the port number in the URL to point to other
>managed server. It looks like second managed server has no idea of
>that session, my app fails because of this.
>
>
>Could you please help me out in this, Do I need to do any thing in
>addition to the above. I couldn't find much in the BEA manual..
>
>
>Thanks
>Rao
-
Remote handle caching error when using HTTP session replication
Using WL 7.0 Sp2 on Solaris, deployed apllication on a 2 instance wide cluster.Code
attached -
unzip testapp.zip to find a test.jar which is the stateless sesion ejb and test.war
which is the applicaiton.
I have this class - TestDelegate that looks up a stateless session EJB(TestEJB),
and puts the remote handle as a member variable(Test bean).Test.jsp instantiates
this delegate(TestDelegate ), and then via setAttribute puts the TestDelegate
in the HTTPSession.Then it calls a couple of methods via the TestDelegate which
has the remote handle of the TestEJB.
Scenario 1 : We deploy the WAR on cluster A which is 2 instance wide,we deploy
the TestEJB on
cluster B, which is also 2 instance wide. All works fine, the delegate is able
to lookup EJB and
can add the remote handle as a member variable.
Scenario 2: We deploy the WAR on cluster A which is 2 instance wide,we deploy
the TestEJB on
cluster B, which is also 2 instance wide. This time we put a weblogic.xml in the
WAR which has tag for in-memory session replication.This time we get class cast
exception for the same sequence that we have above.
<Mar 24, 2003 10:43:28 AM EST> <Error> <kernel> <000802> <ExecuteRequest failed
java.lang.ClassCastException: Assigning instance of class com.TestEJB_1jsmq_EOImpl_WLStub
to field com.bean.TestDelegate#bean
Start server side stack trace:
java.lang.ClassCastException: Assigning instance of class com.TestEJB_1jsmq_EOImpl_WLStub
to field com.bean.TestDelegate#bean
at java.io.ObjectInputStream.inputClassFields(ObjectInputStream.java:2266)
at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:514)
at java.io.ObjectInputStream.inputObject(ObjectInputStream.java:1407)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:381)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:231)
at java.util.Hashtable.readObject(Hashtable.java:794)
at java.lang.reflect.Method.invoke(Native Method)
at java.io.ObjectInputStream.invokeObjectReader(ObjectInputStream.java:2209)
at java.io.ObjectInputStream.inputObject(ObjectInputStream.java:1406)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:381)
at java.io.ObjectInputStream.inputClassFields(ObjectInputStream.java:2258)
at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:514)
at java.io.ObjectInputStream.inputObject(ObjectInputStream.java:1407)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:381)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:231)
at weblogic.common.internal.ChunkedObjectInputStream.readObject(ChunkedObjectInputStream.java:140)
at weblogic.rjvm.MsgAbbrevInputStream.readObject(MsgAbbrevInputStream.java:91)
at weblogic.cluster.replication.ReplicationManager_WLSkel.invoke(Unknown
Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:362)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:313)
at weblogic.security.service.SecurityServiceManager.runAs(SecurityServiceManager.java:821)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:308)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:213)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:189)
End server side stack trace
[testapp.zip]
To see the list of session IDs in the cache you can iterate over the keys of the cache returned from the following code:
NamedCache cacheCatalog = CacheFactory.getReplicatedCacheService("$FilterService$").ensureCache("CoherenceSession.CATALOG", getClass().getClassLoader());Then to see the contents of a session you can take a look a the cache returned from the following code:
String sId = // keys from cacheCatalog
NamedCache cacheDetails = cacheCatalog.getCacheService().ensureCache(sId, getClass().getClassLoader());Hope this helps.
We have received your request for a development license and we hould have ti out to you today.
Later,
Rob Misek
Tangosol, Inc.
Coherence: Cluster your Work. Work your Cluster. -
Strange errors in session replication
I get following errors in session replication :
weblogic.utils.NestedError: Tried to update secondary, but it thought
it was the primary
- with nested exception:
[weblogic.cluster.replication.BadStatusException: updateSecondary
found
-148099416498695151 but it is not the primary]
Using Weblogic451, SP7 on Solaris 2.6.
Any clues ??
-Kuntal Shah.
Hello,
> > Alteon is set to use "permanent hashing" to route the client into
> > the same server in each request.
>
> How do you expect the switch to know about the secondary node
> if the primary fails? This information is stored in the URL and
> parsed by a proxy.
It took a couple of days and fair amount of documentation to understand
the behaviour of the web cluster...;-)
We changed the configuration to include two proxy servers between the
Alteon switch and web cluster, but we ended up with a different problem.
The setup at the moment is:
- Sun Solaris 2.6, native threads, JDK 1.1.7B_008
- two WLS 4.51 SP7 running in cluster using in-memory replication
- only servlets deployed in the servers
- Two WLS proxies between the webcluster and alteon switch. Both proxies are
configured to use both of the web servers. they do not belong to the web
cluster.
The proxies are configured in the following way:
weblogic.httpd.session.enable=true
weblogic.httpd.session.timeoutSecs=1800
weblogic.httpd.session.invalidationIntervalSecs=600
weblogic.httpd.session.URLRewriting.enable=true
weblogic.httpd.session.cookies.enable=true
weblogic.httpd.session.cookie.maxAgeSecs=-1
weblogic.httpd.session.cookie.domain=some.domain.name
weblogic.httpd.register.cluster=weblogic.servlet.internal.HttpClusterServlet
weblogic.httpd.initArgs.cluster=defaultServers=x.y.z.a1:9001|x.y.z.a2:9001
weblogic.httpd.defaultServlet=cluster
weblogic.allow.execute.weblogic.servlet=everyone
Probably most of the configuration parameters are not necessary, can
somebody tell me which are
and which are not?
The servers in the web cluster are configured in the following way:
The following parameters are in the start-up script of the servers:
-Dweblogic.cluster.enable=true \
-Dweblogic.system.name=somehost \
-Dweblogic.cluster.name=someclustercluster \
-Dweblogic.cluster.multicastAddress=237.0.0.1 \
-Dweblogic.system.bindAddr=x.y.z.a1 \
-Dweblogic.system.nativeIO.enable=true \
-Dweblogic.debug.replication=1 \
And the following are in the cluster-wide properties file:
weblogic.httpd.clustering.enable=true
weblogic.httpd.session.persistence=true
weblogic.httpd.session.persistentStoreType=replicated
weblogic.httpd.session.enable=true
weblogic.httpd.session.timeoutSecs=1800
weblogic.httpd.session.invalidationIntervalSecs=600
weblogic.httpd.session.URLRewriting.enable=true
weblogic.httpd.session.cookies.enable=true
weblogic.httpd.session.cookie.domain=some.domain.name
The IP-address of the Alteon box in front of the proxies is bound to
'some.domain.name'.
The first problem seems to be very poor performance. The overall throughtput
is vveeeerrryyy slow, comparing
to the configuration where there is no proxy sitting in front of the
cluster. Furthermore, at some stage the proxy
starts to print out error messages: java.lang.InvalidStateException: broken
pipe [...] . After those exceptions no
client get any response anymore.
Could someone give some hints what are the most essential items in the
weblogic.properties -file for the
proxy server? Also, if someone has some experience from setting up a
configuration similar to ours: one hardware
load-balancer in front of two proxies, and eventually two web servers
running in a web cluster.
Cheers,
Jarno.
-
Session replication not working with OAM clusters
Hi,
We have a enterprise deployment of OAM 11.1.1.5.0, and we are using a cluster or OAM managed servers. Server#A (fano1) hosts the admin + OAM managed server, server#B (fano2) hosts the second OAM managed server.
What we have noticed in general is that when accessing the OAM admin console (on server#A, fano1), we get prompted to login and then we get a "Error 404--Not Found". It seems that we are getting authenticated against server#B (fano2) and our session is not getting replicated to server#A (fano1). After exactly 60 seconds, if we were to simply reload the URL in the same browser window, we can get in. This is with "database replication of sessions" enabled. If I disable that, then I will never be able to login to the admin console.
The admin server logs contain the error message:
[2011-12-22T16:41:52.471-05:00] [AdminServer] [WARNING] [] [oracle.oam.agent-default] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: f4a544324f1d0f29:2ccc59c7:13467aaf744:-8000-0000000000000182,0] [APP: oam_admin#11.1.1.3.0] OAM Server request failed: OpCode = 19 [GetSessionInfo], Returned Status = Major code: 71(SessionInvalid) Minor code: 2(NoCode)
[2011-12-22T16:41:52.473-05:00] [AdminServer] [WARNING] [OAMAGENT-00406] [oracle.oam.agent-default] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: f4a544324f1d0f29:2ccc59c7:13467aaf744:-8000-0000000000000182,0] [APP: oam_admin#11.1.1.3.0] Invalid authentication token: AccessService INVALID_USER_SESSION
One other point is that we are using virtual IPs and each server (Admin + 2 managed servers) is listening to its own virtual IP.
Does anyone have any suggestions on what this could be. My thinking is that coherence based session replication is not working. However I have no way to say that for sure. Is there any way to enable some more logging from either OAM or Coherence that would help verify if sessions are getting replicated or not? I have tried putting the entire system in TRACE:32 but that generates too much data.
Thanks
Aspi Engineer
Putnam InvestmentsDid you get solution to this one.
ThanksKrishna -
Session replication in Clustered Env with Apache proxy
Hi,
Setup:
1. Apache proxy configured using mod_proxy
2. Cluster is set up at weblogic with 2 nodes(on same machine)
3. EAR is deployed on cluster with weblogic-application.xml as below
<session-descriptor>
<persistent-store-type>replicated_if_clustered</persistent-store-type>
<sharing-enabled>true</sharing-enabled>
</session-descriptor>
use case:
user1 hits the URL, Apache proxy forwards to node1
user2 hits the URL, Apache proxy forwards to node2
kill the node1 and user1 click on any link, he gets redirected to login page. No failover us happening.
In Node 1 server logs am getting below exception
####<Dec 5, 2012 8:39:21 PM IST> <Error> <Cluster> <Win2k8SRV-01> <win2k8svr-ms1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1354720161810> <BEA-000126> <All session objects should be serializable to replicate. Check the objects in your session. Failed to replicate non-serializable object.
java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
java.lang.ClassNotFoundException: Failed to load class com.presentation.physical.management.form.ApprovalForm
at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:234)
at weblogic.rmi.internal.BasicRemoteRef.invoke(BasicRemoteRef.java:223)
at weblogic.cluster.replication.ReplicationManager_1035_WLStub.create(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
Strange thing i noticed is if comment the persistent-store-type attribute in xml file then apache is able to load balance the request between the nodes else all request goes to Node1( from apache logs). Below logs are captured with persistent-store-type set to replicated_if_clustered. Here the JSESSIONID points to server1(port 7003 which is appended to JSESSIONID).
192.168.2.161 - - [05/Dec/2012:20:38:52 +0530] JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; path=/; HttpOnly Test=Test "GET /App/login.jsp?partnerApp=http://192.168.2.161:8080/login.do?method=login HTTP/1.1" 200 13384
192.168.2.161 - - [05/Dec/2012:20:38:55 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1 "POST /App/AppEnv HTTP/1.1" 200 -
192.168.2.161 - - [05/Dec/2012:20:39:14 +0530] APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc; path=/ Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1 "POST /App/login HTTP/1.1" 302 289
192.168.1.220 - - [05/Dec/2012:20:39:14 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:14 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "GET /login.do?method=login HTTP/1.1" 200 2767
192.168.1.220 - - [05/Dec/2012:20:39:20 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:20 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "GET /login.do?method=showCorporateLogo HTTP/1.1" 200 11547
192.168.2.161 - - [05/Dec/2012:20:39:20 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /EkaCommonUtilServlet?method=getUserLocaleDate HTTP/1.1" 200 23
192.168.1.220 - - [05/Dec/2012:20:39:20 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:20 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /login.do?method=getMenuData HTTP/1.1" 200 9615
192.168.1.220 - - [05/Dec/2012:20:39:20 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:20 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /getUserModuleHomePage.do?method=loadUserHomePageDetails HTTP/1.1" 200 1264
192.168.2.161 - - [05/Dec/2012:20:39:20 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /ListingHelperServlet?actionId=getUserPref&gridViewName=DEFAULT%20VIEW&gridId=LOC HTTP/1.1" 200 2228
192.168.2.161 - - [05/Dec/2012:20:39:20 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /ListingHelperServlet?actionId=getUserPref&gridViewName=DEFAULT1%20VIEW&gridId=LOCA HTTP/1.1" 200 3775
192.168.1.220 - - [05/Dec/2012:20:39:20 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:20 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /app/commonListing.do?method=getListingData&gridId=LOC HTTP/1.1" 302 309
192.168.1.220 - - [05/Dec/2012:20:39:21 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:20 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /app/contractApprovalList.do?method=loadListOfContractApprovals HTTP/1.1" 302 309
192.168.1.220 - - [05/Dec/2012:20:39:21 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:21 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "GET /private/jsp/AppErrorPage.jsp HTTP/1.1" 200 8042
192.168.1.220 - - [05/Dec/2012:20:39:21 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:21 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "GET /private/jsp/AppErrorPage.jsp HTTP/1.1" 200 8042
192.168.1.220 - - [05/Dec/2012:20:39:24 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:24 +0530] JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE; path=/; HttpOnly Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /app/gmcTabHandler.do?method=loadGMCTab&applyDefaultValue=Y&gridId=LOGPF&linkName=Grower_Profile&switchMenu=TabMenu17_1_1 HTTP/1.1" 200 47435
192.168.1.220 - - [05/Dec/2012:20:39:24 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:24 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /getUserModuleHomePage.do?method=saveUserViewportPreference HTTP/1.1" 200 -
192.168.2.161 - - [05/Dec/2012:20:39:25 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /EkaCommonUtilServlet?method=getUserLocaleDate HTTP/1.1" 200 23
192.168.1.220 - - [05/Dec/2012:20:39:25 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.1.220 - - [05/Dec/2012:20:39:25 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:25 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /login.do?method=getMenuData HTTP/1.1" 200 9615
192.168.2.161 - - [05/Dec/2012:20:39:25 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "GET /login.do?method=showCorporateLogo HTTP/1.1" 200 11547
192.168.1.220 - - [05/Dec/2012:20:39:31 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:25 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /app/commonListing.do?method=getCommonListingPage&gridId=LOGPF&setupPage=Y&applyDefaultValue=Y HTTP/1.1" 200 51216
192.168.1.220 - - [05/Dec/2012:20:39:31 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:31 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /commonListing.do?method=getUserViews HTTP/1.1" 200 49
192.168.1.220 - - [05/Dec/2012:20:39:31 +0530] - JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE!1354720133917 "GET /App/validateSession?ssoSessionID=APP-728a8af1-ME5hYqMOkJCcIQc HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:39:31 +0530] - Test=Test; JSESSIONID=1XZxQ1jFxpMhLr8ZSyJdGWLk6Fn2yvWrmc3G6YZpyDBDSZQGQJTn!-1288095560!-1062731300!7003!-1!NONE; APP_SSO_COOKIE=APP-728a8af1-ME5hYqMOkJCcIQc "POST /app/growerProfile.do?method=loadGrowerProfileFilter&gridId=LOGPF HTTP/1.1" 200 46579
192.168.2.161 - - [05/Dec/2012:20:40:04 +0530] - Test=Test; JSESSIONID=DMBQQ1QSQfh6HyCMH91P2wXJQVKDQr6hCklqJgbhyYlVdzNT6vQC!1022793105!-1062731300!7004!-1; App=App; App2=App2 "GET / HTTP/1.1" 302 397
192.168.2.161 - - [05/Dec/2012:20:40:04 +0530] JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; path=/; HttpOnly Test=Test; JSESSIONID=DMBQQ1QSQfh6HyCMH91P2wXJQVKDQr6hCklqJgbhyYlVdzNT6vQC!1022793105!-1062731300!7004!-1; App=App; App2=App2 "GET /App/login.jsp?partnerApp=http://192.168.2.161:8080/login.do?method=login HTTP/1.1" 200 13384
192.168.2.161 - - [05/Dec/2012:20:40:04 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/css/styles.css HTTP/1.1" 200 4988
192.168.2.161 - - [05/Dec/2012:20:40:04 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/js/ext/resources/css/ext-all.css HTTP/1.1" 200 28527
192.168.2.161 - - [05/Dec/2012:20:40:04 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/js/general/tooltip.js HTTP/1.1" 200 12408
192.168.2.161 - - [05/Dec/2012:20:40:04 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /App/private/js/ForgotPassword.js HTTP/1.1" 200 6003
192.168.2.161 - - [05/Dec/2012:20:40:04 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/js/general/AjaxRequest.js HTTP/1.1" 200 4434
192.168.2.161 - - [05/Dec/2012:20:40:04 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/js/ext/adapter/ext/ext-base.js HTTP/1.1" 200 12483
192.168.2.161 - - [05/Dec/2012:20:40:04 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/js/ext/ext-all.js HTTP/1.1" 200 186157
192.168.2.161 - - [05/Dec/2012:20:40:05 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/images/login.gif HTTP/1.1" 200 427
192.168.2.161 - - [05/Dec/2012:20:40:05 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/images/solutions.gif HTTP/1.1" 200 3492
192.168.2.161 - - [05/Dec/2012:20:40:05 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/images/logo.gif HTTP/1.1" 200 1751
192.168.2.161 - - [05/Dec/2012:20:40:05 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "POST /App/AppEnv HTTP/1.1" 200 -
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET / HTTP/1.1" 302 397
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /App/login.jsp?partnerApp=http://192.168.2.161:8080/login.do?method=login HTTP/1.1" 200 13384
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/css/styles.css HTTP/1.1" 200 4988
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/js/general/AjaxRequest.js HTTP/1.1" 200 4434
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/js/general/tooltip.js HTTP/1.1" 200 12408
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/js/ext/resources/css/ext-all.css HTTP/1.1" 200 28527
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /App/private/js/ForgotPassword.js HTTP/1.1" 200 6003
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/js/ext/adapter/ext/ext-base.js HTTP/1.1" 200 12483
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/js/ext/ext-all.js HTTP/1.1" 200 186157
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/images/login.gif HTTP/1.1" 200 427
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/images/logo.gif HTTP/1.1" 200 1751
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "GET /private/images/solutions.gif HTTP/1.1" 200 3492
192.168.2.161 - - [05/Dec/2012:20:40:46 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "POST /App/AppEnv HTTP/1.1" 200 -
192.168.2.161 - - [05/Dec/2012:20:40:57 +0530] APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp; path=/ Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2 "POST /App/login HTTP/1.1" 302 289
192.168.1.220 - - [05/Dec/2012:20:40:57 +0530] - JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1!1354720204538 "GET /App/validateSession?ssoSessionID=APP-728a8af1-1OOOch71SSRAKYp HTTP/1.1" 200 89
192.168.2.161 - - [05/Dec/2012:20:40:57 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /login.do?method=login HTTP/1.1" 200 2767
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/css/sample.css HTTP/1.1" 200 303
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/css/styles.css HTTP/1.1" 200 4988
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/js/ext/adapter/ext/ext-base.js HTTP/1.1" 200 12483
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/css/portal.css HTTP/1.1" 200 287
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/css/header.css HTTP/1.1" 200 411
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/js/ext/resources/css/ext-all.css HTTP/1.1" 200 28527
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/js/tag/Tag.js HTTP/1.1" 200 3579
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/js/ext/ext-all.js HTTP/1.1" 200 186157
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/js/general/ProgressBarPager.js HTTP/1.1" 200 1244
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/js/general/PanelResizer.js HTTP/1.1" 200 649
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/js/general/PagingMemoryProxy.js HTTP/1.1" 200 1154
192.168.2.161 - - [05/Dec/2012:20:41:00 +0530] - Test=Test; JSESSIONID=b4GBQ1jMpYQTRTbT213yvj1T13Jm6wnh1VlTz79jJWRkXj88jShg!-1288095560!-1062731300!7003!-1!1586878909!-1062731300!7004!-1; App=App; App2=App2; APP_SSO_COOKIE=APP-728a8af1-1OOOch71SSRAKYp "GET /private/js/general/CalendarPopup.js HTTP/1.1" 200 14572
Let me know if my set up wrong. Do i need to modify any settings related to cluster from console ?
Regards,
SandeepHi Mukesh,
Thanks for the reply. TO check the session data i took the JSP from http://middlewaremagic.com/weblogic/?p=290 for testing the serialization of session data, modified a little bit and added to all the WAR files to check session data of each WAR file. I could verify that all data in session are serializable.
While trying the serialization test, i did change the persistence store type to "replicated" but no luck yet.
But this time i got some replication missed message from server logs.
Primary server Logs
<BEA-000118> <Lost 11 replication updates of object 3736767543476998706; update version: 12, current version: 1.>
<BEA-000000> <[roid:3736767543476998706] Found secondary for application key /app>
<BEA-000000> <[roid:3736767543476998706] Found secondary for application key >
<BEA-000000> <[roid:3736767543476998706] Found secondary for application key /ssoApp>
<BEA-000000> <Updated local secondary with version 18 from 3736767543476998705S:192.168.1.220:[7111,7111,-1,-1,-1,-1,-1]:Domain7001:wlsc1s1for replication object 3736767543476998706, key , ro = weblogic.servlet.internal.session.ReplicatedSessionData@14e52305>
<BEA-000000> <Updated local secondary with version 19 from 3736767543476998705S:192.168.1.220:[7111,7111,-1,-1,-1,-1,-1]:Domain7001:wlsc1s1for replication object 3736767543476998706, key , ro = weblogic.servlet.internal.session.ReplicatedSessionData@14e52305>
But in secondary server i got lot of weblogic.cluster.replication.NotFoundException
<BEA-000000> <Has secondary servers? true>
<BEA-000000> <Current secondary server? 3054493844538055224S:192.168.1.220:[8111,8111,-1,-1,-1,-1,-1]:Domain7001:wlsc1s2>
<BEA-000000> <[roid:3736767543476998706] Created secondary on 3054493844538055224S:192.168.1.220:[8111,8111,-1,-1,-1,-1,-1]:Domain7001:wlsc1s2>
<BEA-000000> <[roid:3736767543476998706] Using Single-Channel for 2-way update()>
<BEA-000000> <[roid:3736767543476998706] Error updating secondary for 3736767543476998706, key /app, on 3054493844538055224S:192.168.1.220:[8111,8111,-1,-1,-1,-1,-1]:Domain7001:wlsc1s2. Re-creating secondary.
weblogic.cluster.replication.NotFoundException: Lost 8 updates of 3736767543476998706
at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:234)
at weblogic.rmi.internal.BasicRemoteRef.invoke(BasicRemoteRef.java:223)
at weblogic.cluster.replication.ReplicationManager_1035_WLStub.update(Unknown Source)
at sun.reflect.GeneratedMethodAccessor252.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at weblogic.cluster.replication.SecureReplicationInvocationHandler$ReplicationServicesInvocationAction.run(SecureReplicationInvocationHandler.java:194)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.cluster.replication.SecureReplicationInvocationHandler.invoke(SecureReplicationInvocationHandler.java:164)
at $Proxy359.update(Unknown Source)
at weblogic.cluster.replication.ReplicationManager.sendUpdateRequestToSecondary(ReplicationManager.java:740)
at weblogic.cluster.replication.ReplicationManager.updateSecondary(ReplicationManager.java:662)
at weblogic.servlet.internal.session.ReplicatedSessionData.syncSession(ReplicatedSessionData.java:639)
at weblogic.servlet.internal.session.ReplicatedSessionContext.sync(ReplicatedSessionContext.java:85)
at weblogic.servlet.internal.ServletRequestImpl$SessionHelper.syncSession(ServletRequestImpl.java:2860)
at weblogic.servlet.internal.ServletRequestImpl$SessionHelper.syncSession(ServletRequestImpl.java:2835)
at weblogic.servlet.internal.ServletResponseImpl$1.run(ServletResponseImpl.java:1485)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.internal.ServletResponseImpl.send(ServletResponseImpl.java:1479)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1462)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
Caused By: weblogic.cluster.replication.NotFoundException: Lost 8 updates of 3736767543476998706
at weblogic.cluster.replication.ReplicationManager.updateInternal(ReplicationManager.java:1143)
at weblogic.cluster.replication.ReplicationManager.update(ReplicationManager.java:1101)
at weblogic.cluster.replication.ReplicationManager_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:667)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:522)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:518)
at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
From http://docs.oracle.com/cd/E28389_01/apirefs.1111/e14397/Cluster.html the exception is thrown if secondary is unable to receive the updates and asking primary to recreate the session state(BEA-000118).
Am also getting
<BEA-000000> <[roid:3736767543476998706] Removing secondary for key >
<BEA-000000> <Removed secondary for roids: 3736767543476998706>
in primary server logs does this mean primary is creating and deleting the session state on secondary ?
Am getting below assertion error in primary server logs
<BEA-080004> <An error was thrown by rmi server: weblogic.cluster.leasing.databaseless.RMIClusterMessageEndPointImpl.process(Lweblogic.cluster.leasing.databaseless.ClusterMessage;)
java.lang.AssertionError: Invalid state transition from stable_leader to stable.
java.lang.AssertionError: Invalid state transition from stable_leader to stable
at weblogic.cluster.leasing.databaseless.ClusterState.setState(ClusterState.java:100)
at weblogic.cluster.leasing.databaseless.ClusterState.setState(ClusterState.java:59)
at weblogic.cluster.leasing.databaseless.ClusterMember.handleJoinResponseRequest(ClusterMember.java:224)
at weblogic.cluster.leasing.databaseless.ClusterMember.process(ClusterMember.java:102)
at weblogic.cluster.leasing.databaseless.RMIClusterMessageEndPointImpl.process(RMIClusterMessageEndPointImpl.java:30)
at weblogic.cluster.leasing.databaseless.RMIClusterMessageEndPointImpl_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:667)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:522)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:518)
at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
>
After killing the primary and access the application in secondary server am getting unable to create secondary on null logs like below. Am i missing any configuration in setup ?
<Managed server wlsc1s2 has been suspended or shutdown.>
<BEA-000129> <Removing 3054493844538055224S:192.168.1.220:[8111,8111,-1,-1,-1,-1,-1]:Domain7001:wlsc1s2 from the cluster.>
<BEA-000000> <Has secondary servers? true>
<Current secondary server? null>
<BEA-000000> <Preferred list : []>
<BEA-000000> <Remote list : []>
<BEA-000000> <Local list : []>
<New secondary server is null>
<BEA-000000> <Secondary server null>
<BEA-000000> <Changed the status of 2 objects and it took 59 ms>
<BEA-000000> <Has secondary servers? true>
<BEA-000000> <Current secondary server? null>
<BEA-000000> <Preferred list : []>
<BEA-000000> <Remote list : []>
<BEA-000000> <Local list : []>
<BEA-000000> <New secondary server is null>
<BEA-000000> <[roid:3054493844538055225] Unable to create secondary on null>
<BEA-000000> <[roid:3054493844538055225] Unable to create secondary on null>
Regards,
sandeep -
WS7 cluster session replication
Now that my WS7 cluster is up and running, I'm attempting to deploy a simple JSF webapp that incorporates session replication between the nodes in my cluster.
I've followed the instructions in the WS7 admin guide to enable session replication, and I've added the following to my sun-web.xml deployment descriptor:
<session-config>
<session-manager persistence-type="replicated"/>
</session-config>The first problem I discovered was that my JSF managed bean was not serializable, so I fixed that. Now, however, when I load the initial page in my app, I am getting an NPE that looks like it is coming from within the WS7 session replication code:
[23/Dec/2009:08:46:51] warning ( 3692): for host 172.16.2.47 trying to GET /sessionTest/index.jsf while trying to GET /sessionTest/, service-j2ee reports: executePhase(RENDER_RESPONSE 6,com.sun.faces.context.FacesContextImpl@2de69e99) threw exception
javax.faces.FacesException
at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:135)
at com.sun.faces.lifecycle.LifecycleImpl.phase(LifecycleImpl.java:251)
at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:144)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:245)
at org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:398)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:277)
at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:255)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:586)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:556)
at com.sun.web.replication.session.SessionLockingStandardPipeline.invoke(SessionLockingStandardPipeline.java:71)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:187)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:586)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:556)
at com.sun.webserver.connector.nsapi.NSAPIProcessor.service(NSAPIProcessor.java:160)
Caused by: ClientAbortException: java.io.IOException: WEB8001: Write failed
at org.apache.coyote.tomcat5.OutputBuffer.realWriteBytes(OutputBuffer.java:385)
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:404)
at org.apache.coyote.tomcat5.OutputBuffer.doFlush(OutputBuffer.java:334)
at org.apache.coyote.tomcat5.OutputBuffer.flush(OutputBuffer.java:313)
at org.apache.coyote.tomcat5.CoyoteResponse.flushBuffer(CoyoteResponse.java:622)
at com.sun.webserver.connector.nsapi.NSAPIResponse.flushBuffer(NSAPIResponse.java:127)
at org.apache.coyote.tomcat5.CoyoteResponseFacade.flushBuffer(CoyoteResponseFacade.java:291)
at com.sun.faces.application.ViewHandlerImpl.renderView(ViewHandlerImpl.java:203)
at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:106)
... 14 more
Caused by: java.io.IOException: WEB8001: Write failed
at com.sun.webserver.connector.nsapi.NSAPIResponseStream.doWrite(NSAPIResponseStream.java:61)
at org.apache.coyote.Response.doWrite(Response.java:575)
at org.apache.coyote.tomcat5.OutputBuffer.realWriteBytes(OutputBuffer.java:380)
... 22 more
[23/Dec/2009:08:46:51] warning ( 3692): for host 172.16.2.47 trying to GET /sessionTest/index.jsf while trying to GET /sessionTest/, service-j2ee reports: REPL0080: Received exception while reading session from store [{0}]
java.lang.NullPointerException
at com.sun.web.replication.session.LWSFPersistentManager.doRead(LWSFPersistentManager.java:162)
at com.sun.web.replication.session.LWSFValve.invoke(LWSFValve.java:31)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:556)
at com.sun.web.replication.session.SessionLockingStandardPipeline.invoke(SessionLockingStandardPipeline.java:71)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:187)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:586)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:556)
at com.sun.webserver.connector.nsapi.NSAPIProcessor.service(NSAPIProcessor.java:160)I thought that perhaps once the session was created, I would stop seeing this error. It isn't exactly the same, but I am still seeing an NPE even after the session has been created:
[23/Dec/2009:08:47:33] warning ( 3692): for host 172.16.2.47 trying to POST /sessionTest/index.jsf, service-j2ee reports: REPL0080: Received exception while reading session from store [{0}]
java.lang.NullPointerException
at com.sun.web.replication.session.LWSFPersistentManager.doRead(LWSFPersistentManager.java:162)
at com.sun.web.replication.session.LWSFValve.invoke(LWSFValve.java:31)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:556)
at com.sun.web.replication.session.SessionLockingStandardPipeline.invoke(SessionLockingStandardPipeline.java:71)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:187)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:586)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:556)
at com.sun.webserver.connector.nsapi.NSAPIProcessor.service(NSAPIProcessor.java:160)Any idea what's going wrong? Is it something I've done?
I can give you the code for my webapp if you need it.
Thanks,
Bill1. First try deploying your application without having this parameter, This is need only when you need session replication at web application level not at web server level..
2. Try with this option as well:
<sun-web-app>
<session-config>
<session-manager persistence-type="replicated">
</session-manager>
</session-config>
</sun-web-app> -
CF7 Clustering - session replication broken
Further to my
earlier
thread on CF7 clustering and the "network error",
I think session replication is broken. True, I can't find any
documentation which explains how it is supposed to behave, but this
is what I'm seeing:
If you have a two node cluster, and person A has a session on
node 1 and person B a session on node 2, session variables for each
user DO NOT appear on both nodes
Sessions on one node will only appear on the other when the
first node fails over
Moreover, they will only appear when the user makes another
request
CFLOGIN, when using loginstorage="session" does not replicate.
This means you have to use sticky sessions, and if there is a node
failure, the user will have to login in again. This can be avoided
by using loginstorage = "cookie" but this means the username and
password are stored unencrypted in the cookie.
Problems I have with this:
It's impossible to have a cluster wide view of sessions. E.g.
you can't count how many sessions you have at any one time. Any
code that makes use of being able to dig in to multiple sessions -
made possible by J2EE sessions - will not work reliably.
CFLOGIN is broken. It would be better to not use it for sites
that have to be secure as keeping the unencrypted username and
password in a cookie is very insecure. The only way to protect the
username and password being stored in the cookie is SSL; and that
means that every single item on the site needs to be protected by
SSL to stop the browser from serving up the cookie over an
unprotected connection.
It's not clear what is responsible for replication. If sessions
are only replicated when an instance goes down, rather than
continously, then I assume that the master instance of CF on each
server is responsible for replication. What happens if the server
crashes?Information about session replication can be found here: http://docs.oracle.com/cd/E21764_01/web.1111/e13709/failover.htm#i1024620
it contains a section 'Requirements for HTTP Session State Replication' that might be helpful to you.
Also important to note is, how the load balancer handles session binding and the WebLogic cookie
http://docs.oracle.com/cd/E21764_01/web.1111/e13709/load_balancing.htm#i1026939 -
Session Replication doesn't work when using a custom Unicast Channel
Hello!
After configure a WLS Cluster for an WebApp with session replication support enabled I faced some issues with cluster configuration.
My LAB env used for this configurations is:
One Solaris 10 SPARC box.
-- One WLS 11g (10.3.6) domain with:
---- 4 Managed servers:
---- Admin server
---- server-1
---- server-2
---- Proxy Server (HttpClusterServlet)
--- 1 Cluster composed by:
---- server-1
---- server-2In that setup I noticed if I define a custom network channel for servers ( server>protocols>channels ) in the cluster and set Cluster Messaging Mode as Unicast* in the Cluster config ( Cluster>Configuration>Messaging>Messaging Mode ), so the session state replication does not work.
When I enable the cluster replication debug for managed servers the following messages appears:
<> <> <1358966729933> <BEA-000000> <[roid:-1772481434088297851] Creating primary for application key /webapp>
####<Jan 23, 2013 4:45:29 PM BRST> <Debug> <ReplicationDetails> <de25503> <server-1> <[ACTIVE] ExecuteThread: '5' for queue: > 'weblogic.kernel.Default (self-tuning)'> <<ano
nymous>> <> <> <1358966729958> <BEA-000000> *<Has secondary servers? false>*
####<Jan 23, 2013 4:45:29 PM BRST> <Debug> <ReplicationDetails> <de25503> <server-1> <[ACTIVE] ExecuteThread: '5' for queue: >'weblogic.kernel.Default (self-tuning)'> <<ano
nymous>> <> <> <1358966729959> <BEA-000000> *<Current secondary server? null>*
####<Jan 23, 2013 4:45:29 PM BRST> <Debug> <Replication> <de25503> <server-1> <[ACTIVE] ExecuteThread: '5' for queue: >'weblogic.kernel.Default (self-tuning)'> <<anonymous>
<> <> <1358966729959> <BEA-000000> <[roid:-1772481434088297851] Unable to create secondary on null>
####<Jan 23, 2013 4:45:31 PM BRST> <Debug> <ReplicationDetails> <de25503> <server-1> <[ACTIVE] ExecuteThread: '5' for queue: >'weblogic.kernel.Default (self-tuning)'> After eliminate all possible issues with my webapp (serialization, weblogic descriptor configuration, etc) and try many cluster network configurations I noticed that this problem only occurs when I use Unicast for Cluster's Messaging.
At the end of the day I really would like to understand why the session replication only works for Cluster's Messaging using Multicast mode. I read a lot the WLS docs (specifically the cluster/network topics) [1][2], but I can't find an official explanation about this.
Can someone help me understand this behavior?
Many thanks.
[1] http://docs.oracle.com/cd/E15523_01/web.1111/e13701/network.htm
[2] http://docs.oracle.com/cd/E15523_01/web.1111/e13709/setup.htmI have Fluxbox started with Slim and .xinitrc. Dbus works only with:
exec ck-launch-session startfluxbox
you need run Openbox with ck-launch-session:
exec ck-launch-session openbox-session
Bye!! -
Sun java System Web Server 7.0 up2 session replication setting
hi ....
Following the standard example of reverse proxy and 2 cluster nodes with session replication
I get the following starting message
info ( 3355): CORE5076: Using [Java HotSpot(TM) Server VM, Version 1.5.0_12] from [Sun Microsystems Inc.]
[07/Aug/2008:15:26:45] warning ( 3355): REPL0081: No backup instance configured for replication service. Disabling replication service
[07/Aug/2008:15:26:47] info ( 3355): WEB0100: Loading web module in virtual server [mycluster8087] at [myweb]
[07/Aug/2008:15:26:48] warning ( 3355): WEB9200: sun-web.xml DTD Version with public ID = [-//Sun Microsystems, Inc.//DTD Application Server 8.1 Servlet 2.5//EN] and system ID = [http://www.sun.com/software/appserver/dtds/sun-web-app_2_4-1.dtd] not found in the local respository. Using DTD version with system ID = [http://www.sun.com/software/appserver/dtds/sun-web-app_2_5-0.dtd] instead.
[07/Aug/2008:15:26:48] warning ( 3355): WEB7204: Application [myweb] is configured with persistence-type [replicated] but Session Replication is not enabled on this instance; falling back to persistence-type [memory]
[07/Aug/2008:15:26:49] info ( 3355): PWC3031: Security role name tomcat used in an <auth-constraint> without being defined in a <security-role> in context [myweb]
[07/Aug/2008:15:26:49] info ( 3355): PWC3031: Security role name role1 used in an <auth-constraint> without being defined in a <security-role> in context [myweb]
[07/Aug/2008:15:26:49] info ( 3355): HTTP3072: http-listener-1: http://mycluster8087:8087 ready to accept requests
[07/Aug/2008:15:26:49] info ( 3355): CORE3274: successful server startup
[07/Aug/2008:15:27:37] info ( 3355): CORE5073: Web server shutdown in progress
********** sun-web.xml ***************
<!DOCTYPE sun-web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Application Server
8.1 Servlet 2.5//EN" "http://www.sun.com/software/appserver/dtds/sun-web-app_2_4
-1.dtd">
<sun-web-app>
<session-config>
<session-manager persistence-type="replicated"/>
</session-config>
</sun-web-app>
** path : dir-root/https-[instance-name]/web-app/[instance-name]/myweb/WEB-INF/sun-web.xml setting ...
Why is it a case a session replication doing not become?
Is it a case one a mistake by a setting?Try this blog [http://blogs.sun.com/nsegura/entry/h2_session_replication_and_lightweight|http://blogs.sun.com/nsegura/entry/h2_session_replication_and_lightweight]
4. Enabling Session Replication
Now that we deployed our web application, we need to enable the session replication feature in the configuration. This will start the session replication services in each instance. We can do this using the CLI administration:
wadm> set-session-replication-prop --config=mycluster enabled=true
CLI201 Command 'set-session-replication-prop' ran successfully
wadm> deploy-config mycluster
CLI201 Command 'deploy-config' ran successfully
Maybe you are looking for
-
My entire camera app doesnt work when i try opening it my camera crashed after the first installation. The only place i can take pics is in my photo booth and only front pics from d feont camera
-
Error after running Projector for a while
I run a projector in a kiosk. When being used there is no problem, but at night if it is not being used for a bit I get one of two errors Faulting application dirproj.exe, version 10.1.0.11, faulting module iml32.dll, version 10.1.0.11, fault address
-
After upgrading to Maverick, my computer constantly loses internet connectivity
I have an iMac with 3.06 GHz Intel Core 2 Duo processor, Memory: 4 GB 1067 MHz DDR3, 1 TB storage capacity. Since upgrading to Maverick, my internet seems to lose connectivity constantly. Almost every time I open up Safari and type in a search, I g
-
Epson Artisan 1430 printing problems
When using PSE 9 to control printing using a paper-specific ICC profile (as I am used to doing), all output from my new Epson Artisan 1430 A3+ printer had a green cast. This was not a problem when printing the same images to a Canon A4 printer. After
-
Increase moving average price in a STO process
Dear all, I have a STO (MM/SD) and I need to increase my moving average price in 10 %. So, I added a new condition ZTPC in a standard price procedure RM2000 as attached. My problem is I can only account this condition and increase my moving average p