Member 2 is not starting on websphere cluster with PT8.49 (Mapping clash)

Question

Hi all,
i am using PeopleTools 8.49 and configured websphere 6.1 ND cluster on two machines named(ibmwas1,ibmwas2). when i try to start both the members of the cluster, member1 starts succesfully but member2 does not start. the deployment manager profile is created on ibmwas1. the exception is
************ Start Display Current Environment ************
WebSphere Platform 6.1 [ND 6.1.0.0 b0620.14] running with process name ibmwas1Cell01\hrcs9Node\member1 and process id 2896
Host Operating System is Windows Server 2003, version 5.2 build 3790 Service Pack 1
Java version = J2RE 1.5.0 IBM J9 2.3 Windows Server 2003 x86-32 j9vmwi3223-20060504 (JIT enabled)
J9VM - 20060501_06428_lHdSMR
JIT - 20060428_1800_r8
GC - 20060501_AA, Java Compiler = j9jit23, Java VM name = IBM J9 VM
was.install.root = C:\IBM\WebSphere\AppServer
user.install.root = C:\PT8.49\webserv\hrcs9
Java Home = C:\IBM\WebSphere\AppServer\java\jre
ws.ext.dirs = C:\IBM\WebSphere\AppServer/java/lib;C:\PT8.49\webserv\hrcs9/classes;C:\IBM\WebSphere\AppServer/classes;C:\IBM\WebSphere\AppServer/lib;C:\IBM\WebSphere\AppServer/installedChannels;C:\IBM\WebSphere\AppServer/lib/ext;C:\IBM\WebSphere\AppServer/web/help;C:\IBM\WebSphere\AppServer/deploytool/itp/plugins/com.ibm.etools.ejbdeploy/runtime
Classpath = C:\PT8.49\webserv\hrcs9/properties;C:\IBM\WebSphere\AppServer/properties;C:\IBM\WebSphere\AppServer/lib/startup.jar;C:\IBM\WebSphere\AppServer/lib/bootstrap.jar;C:\IBM\WebSphere\AppServer/lib/j2ee.jar;C:\IBM\WebSphere\AppServer/lib/lmproxy.jar;C:\IBM\WebSphere\AppServer/lib/urlprotocols.jar;C:\IBM\WebSphere\AppServer/deploytool/itp/batchboot.jar;C:\IBM\WebSphere\AppServer/deploytool/itp/batch2.jar;C:\IBM\WebSphere\AppServer/java/lib/tools.jar
Java Library path = C:\IBM\WebSphere\AppServer\java\jre\bin;.;C:\IBM\WebSphere\AppServer\java\jre\bin;C:\IBM\WebSphere\AppServer\bin;C:\IBM\WebSphere\AppServer\java\bin;C:\IBM\WebSphere\AppServer\java\jre\bin;C:\oracle\product\10.2.0\client_1\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem
************* End Display Current Environment *************
[11/5/08 23:09:23:046 PKT] 0000001b WebApp E Error while adding servlet mapping.
java.lang.Exception: Mapping clash for ServletWrapper[Proxy:[Proxy:/PP/*]]: Target com.ibm.ws.portletcontainer.portletserving.PortletServingExtensionProcessor@61fc61fc already exists at node PP
     at com.ibm.ws.util.ClauseNode.add(ClauseNode.java:59)
     at com.ibm.ws.util.URIMatcher.put(URIMatcher.java:131)
     at com.ibm.ws.util.URIMapper.addMapping(URIMapper.java:47)
     at com.ibm.ws.webcontainer.webapp.WebApp.initializeTargetMappings(WebApp.java:449)
     at com.ibm.ws.webcontainer.webapp.WebApp.commonInitializationFinish(WebApp.java:274)
     at com.ibm.ws.wswebcontainer.webapp.WebApp.initialize(WebApp.java:271)
     at com.ibm.ws.wswebcontainer.webapp.WebGroup.addWebApplication(WebGroup.java:88)
     at com.ibm.ws.wswebcontainer.VirtualHost.addWebApplication(VirtualHost.java:157)
     at com.ibm.ws.wswebcontainer.WebContainer.addWebApp(WebContainer.java:653)
     at com.ibm.ws.wswebcontainer.WebContainer.addWebApplication(WebContainer.java:606)
     at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:333)
     at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:549)
     at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1295)
     at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1129)
     at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:567)
     at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:814)
     at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:948)
     at com.ibm.ws.runtime.component.ApplicationMgrImpl$AppInitializer.run(ApplicationMgrImpl.java:2114)
     at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:340)
     at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1498)
[11/5/08 23:09:32:843 PKT] 0000001b WebApp E Error while adding servlet mapping.
java.lang.Exception: Mapping clash for ServletWrapper[PortletContainerServlet:[PortletContainerServlet:/portlet/*]]: Target com.ibm.ws.portletcontainer.webextension.PortletExtensionProcessor@69f269f2 already exists at node portlet
     at com.ibm.ws.util.ClauseNode.add(ClauseNode.java:59)
     at com.ibm.ws.util.URIMatcher.put(URIMatcher.java:131)
     at com.ibm.ws.util.URIMapper.addMapping(URIMapper.java:47)
     at com.ibm.ws.webcontainer.webapp.WebApp.initializeTargetMappings(WebApp.java:449)
     at com.ibm.ws.webcontainer.webapp.WebApp.commonInitializationFinish(WebApp.java:274)
     at com.ibm.ws.wswebcontainer.webapp.WebApp.initialize(WebApp.java:271)
     at com.ibm.ws.wswebcontainer.webapp.WebGroup.addWebApplication(WebGroup.java:88)
     at com.ibm.ws.wswebcontainer.VirtualHost.addWebApplication(VirtualHost.java:157)
     at com.ibm.ws.wswebcontainer.WebContainer.addWebApp(WebContainer.java:653)
     at com.ibm.ws.wswebcontainer.WebContainer.addWebApplication(WebContainer.java:606)
     at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:333)
     at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:549)
     at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1295)
     at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1129)
     at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:567)
     at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:814)
     at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:948)
     at com.ibm.ws.runtime.component.ApplicationMgrImpl$AppInitializer.run(ApplicationMgrImpl.java:2114)
     at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:340)
     at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1498)
any help regarding it will be really appreciated.

Answer 1

in addition i also see the logs,
Index Count Time of last Occurrence Exception SourceId ProbeId
---------------------------------------+--------------------------
1 1 11/6/08 0:44:15:562 PKT com.ibm.websphere.management.exception.AdminException com.ibm.ws.management.nodeagent.NodeAgent.launchProcess 298
---------------------------------------+--------------------------
+ 2 1 11/6/08 0:44:15:671 PKT javax.management.MBeanException com.ibm.ws.management.AdminServiceImpl.invoke 679
+ 3 1 11/6/08 0:44:15:749 PKT java.lang.reflect.InvocationTargetException com.ibm.ws.management.connector.soap.SOAPConnector.invoke 351
+ 4 1 11/6/08 0:44:16:030 PKT javax.management.MBeanException com.ibm.ws.management.connector.soap.SOAPConnector.service 212
Index Count Time of last Occurrence Exception SourceId ProbeId
---------------------------------------+--------------------------
1 1 11/6/08 0:43:51:655 PKT com.ibm.ws.dcs.vri.common.DCSDumpException com.ibm.ws.dcs.vri.common.util.DCSDiagnosticModule.causeDump 100
2 2 11/5/08 23:10:26:827 PKT java.util.zip.ZipException com.ibm.ws.classloader.ClassLoaderUtils.addDependents 238
---------------------------------------+--------------------------
------Start of DE processing------ = [11/6/08 0:43:51:655 PKT] , key = com.ibm.ws.dcs.vri.common.DCSDumpException com.ibm.ws.dcs.vri.common.util.DCSDiagnosticModule.causeDump 100
Exception = com.ibm.ws.dcs.vri.common.DCSDumpException
Source = com.ibm.ws.dcs.vri.common.util.DCSDiagnosticModule.causeDump
probeid = 100
Stack Dump = com.ibm.ws.dcs.vri.common.DCSDumpException: Closing the connection because this member did not manage to connect with ibmwas1Cell01\ibmwas2Node01\nodeagent
     at com.ibm.ws.dcs.vri.common.Globals.causeDump(Globals.java:294)
     at com.ibm.ws.dcs.vri.transportAdapter.rmmImpl.ptpDiscovery.DiscoveryServerMgr.reportPing(DiscoveryServerMgr.java:1280)
     at com.ibm.ws.dcs.vri.transportAdapter.rmmImpl.ptpDiscovery.DiscoveryRcv.doMessage(DiscoveryRcv.java:521)
     at com.ibm.ws.dcs.vri.transportAdapter.rmmImpl.ptpDiscovery.DiscoveryRcv.access$600(DiscoveryRcv.java:54)
     at com.ibm.ws.dcs.vri.transportAdapter.rmmImpl.ptpDiscovery.DiscoveryRcv$DiscoverRcvMsgJob.process(DiscoveryRcv.java:690)
     at com.ibm.ws.dcs.vri.common.JobsProcessorThread.executeJob(JobsProcessorThread.java:411)
     at com.ibm.ws.dcs.vri.common.JobsProcessorThread.run(JobsProcessorThread.java:281)
Dump of callerThis =
Object type = com.ibm.ws.dcs.vri.common.util.DCSDiagnosticModule
com.ibm.ws.dcs.vri.common.util.DCSDiagnosticModule@5560556
==> Performing default dump from com.ibm.ws.dcs.vri.common.util.DCSDiagnosticModule = Thu Nov 06 00:43:51 PKT 2008
Reason = com.ibm.ws.dcs.vri.common.DCSDumpException: Closing the connection because this member did not manage to connect with ibmwas1Cell01\ibmwas2Node01\nodeagent
******************* DiscoveryServerMgr::dump() *******************
defined node listeners: PtpRmmNode, PtpRmmNode|ibmwas1Cell01\hrcs9Node\nodeagent|null|1=[DefaultCoreGroup], PtpRmmNode|ibmwas1Cell01\ibmwas2Node01\nodeagent|UnicastConnection@529538960, address: 127.0.0.1:9353 | 2855, isOutbound: true, isValid: true (false false).|1=[DefaultCoreGroup], PtpRmmNode|ibmwas1Cell01\hrcs9Node\server1|null|1=[DefaultCoreGroup], PtpRmmNode|ibmwas1Cell01\ibmwas1CellManager01\dmgr|UnicastConnection@1635541372, address: 192.168.3.51:9352 | 2017, isOutbound: true, isValid: true (false false).|1=[DefaultCoreGroup]}connected nodes: PtpRmmNodeconnected nodes: PtpRmmNodedefined node listeners: PtpRmmNode, PtpRmmNode|ibmwas1Cell01\hrcs9Node\nodeagent|null|1=[DefaultCoreGroup], PtpRmmNode|ibmwas1Cell01\ibmwas2Node01\nodeagent|UnicastConnection@529538960, address: 127.0.0.1:9353 | 2855, isOutbound: true, isValid: true (false false).|1=[DefaultCoreGroup], PtpRmmNode|ibmwas1Cell01\hrcs9Node\server1|null|1=[DefaultCoreGroup], PtpRmmNode|ibmwas1Cell01\ibmwas1CellManager01\dmgr|UnicastConnection@1635541372, address: 192.168.3.51:9352 | 2017, isOutbound: true, isValid: true (false false).|1=[DefaultCoreGroup]}
Defined Nodes Listeners
Node name: ibmwas1Cell01\hrcs9Node\member1, state: RCV_DEAD_TX_LIVE
Node name: ibmwas1Cell01\hrcs9Node\nodeagent, state: RCV_DEAD_TX_DEAD
Node name: ibmwas1Cell01\ibmwas2Node01\nodeagent, state: RCV_LIVE_TX_LIVE
Node name: ibmwas1Cell01\hrcs9Node\server1, state: RCV_DEAD_TX_LIVE
Node name: ibmwas1Cell01\ibmwas1CellManager01\dmgr, state: CONNECTED
****************** DiscoveryRcv::dump() **************************
_liveRcvNodesStreams:
{ibmwas1Cell01\ibmwas2Node01\member2=com.ibm.ws.dcs.vri.transportAdapter.rmmImpl.ptpDiscovery.DiscoveryRcv$InData@d800d8, ibmwas1Cell01\ibmwas2Node01\nodeagent=com.ibm.ws.dcs.vri.transportAdapter.rmmImpl.ptpDiscovery.DiscoveryRcv$InData@2c422c42, ibmwas1Cell01\ibmwas1CellManager01\dmgr=com.ibm.ws.dcs.vri.transportAdapter.rmmImpl.ptpDiscovery.DiscoveryRcv$InData@2d462d46}
rmm Receiver:
RMMSTATS_
RMMSTATS_********* - 1.9.8.13 - 02/05/06 - *********************
RMMSTATS_UnicastPReceiver Status:
RMMSTATS_ HBProc: r1121. PProc: r6640. connections: 5
RMMSTATS_ RejectedStreamSize: 0
RMMSTATS_------------ QueueR: com.ibm.rmm.mtl.receiver.MStreamSetR@5d365d36
RMMSTATS_ Fifo Q size: 0. Msgs Q size: 0
RMMSTATS_------------ QueueR: com.ibm.rmm.mtl.receiver.MStreamSetR@28162816
RMMSTATS_ Fifo Q size: 0. Msgs Q size: 0
RMMSTATS_ Stream: -4216846273238326136 Name: ibmwas1Cell01\ibmwas1CellManager01\dmgr#DCS#PTP#CON#DefaultCoreGroup#PtpRmmNode|ibmwas1Cell01\ibmwas1CellManager01\dmgr|192.168.3.51|9352
RMMSTATS_ Front #: 155. Trail#: -1. Contig #: 155. Processed Packs: 156
RMMSTATS_ Received Msgs: 156. Delivered Msgs: 156
RMMSTATS_------------ QueueR: com.ibm.rmm.mtl.receiver.MStreamSetR@16f416f4
RMMSTATS_ Fifo Q size: 0. Msgs Q size: 0
RMMSTATS_ Stream: -4217127748215036792 Name: ??????115800ae:11d6d5bc626:-8000ibmwas1Cell01\ibmwas1CellManager01\dmgr#DCS#PTP#DISC#DefaultCoreGroup#PtpRmmNode|ibmwas1Cell01\ibmwas1CellManager01\dmgr|192.168.3.51|9352
RMMSTATS_ Front #: 0. Trail#: -1. Contig #: 0. Processed Packs: 1
RMMSTATS_ Received Msgs: 1. Delivered Msgs: 1
RMMSTATS_ Stream: -2190226440921537397 Name: ??????115800ae:11d6d5bc626:-8000ibmwas1Cell01\ibmwas2Node01\member2#DCS#PTP#DISC#DefaultCoreGroup#PtpRmmNode|ibmwas1Cell01\ibmwas2Node01\member2|192.168.3.52|9355
RMMSTATS_ Front #: 2798. Trail#: 1699. Contig #: 2798. Processed Packs: 2799
RMMSTATS_ Received Msgs: 2799. Delivered Msgs: 2799
RMMSTATS_ Stream: -621566395713051511 Name: ??????115800ae:11d6d5bc626:-8000ibmwas1Cell01\ibmwas2Node01\nodeagent#DCS#PTP#DISC#DefaultCoreGroup#PtpRmmNode|ibmwas1Cell01\ibmwas2Node01\nodeagent|192.168.3.52|9353
RMMSTATS_ Front #: 15. Trail#: -1. Contig #: 15. Processed Packs: 16
RMMSTATS_ Received Msgs: 16. Delivered Msgs: 16
RMMSTATS_ Stream: -623255245573315447 Name: ??????115800ae:11d6d5bc626:-8000ibmwas1Cell01\ibmwas2Node01\nodeagent#DCS#PTP#DISC#DefaultCoreGroup#PtpRmmNode|ibmwas1Cell01\ibmwas2Node01\nodeagent|192.168.3.52|9353
RMMSTATS_ Front #: 14. Trail#: -1. Contig #: 14. Processed Packs: 15
RMMSTATS_ Received Msgs: 15. Delivered Msgs: 15
RMMSTATS_********************************
*********************** DiscoverySubServer::dump() *******************
Stack name: DefaultCoreGroup
Defined:
{ibmwas1Cell01\hrcs9Node\member1=PtpRmmNode|ibmwas1Cell01\hrcs9Node\member1|null|1, ibmwas1Cell01\hrcs9Node\nodeagent=PtpRmmNode|ibmwas1Cell01\hrcs9Node\nodeagent|null|1, ibmwas1Cell01\ibmwas2Node01\nodeagent=PtpRmmNode|ibmwas1Cell01\ibmwas2Node01\nodeagent|UnicastConnection@529538960, address: 127.0.0.1:9353 | 2855, isOutbound: true, isValid: true (false false).|1, ibmwas1Cell01\hrcs9Node\server1=PtpRmmNode|ibmwas1Cell01\hrcs9Node\server1|null|1, ibmwas1Cell01\ibmwas1CellManager01\dmgr=PtpRmmNode|ibmwas1Cell01\ibmwas1CellManager01\dmgr|UnicastConnection@1635541372, address: 192.168.3.51:9352 | 2017, isOutbound: true, isValid: true (false false).|1}
****************** DiscoveryTx::dump() ibmwas1Cell01\ibmwas2Node01\member2 *******************
Attempting nodes:
Dead nodes:
PtpRmmNode
Live nodes:
PtpRmmNode
Rmm transmitter:
RMMSTATS_
RMMSTATS_********* - 1.9.8.13 - 02/05/06 | 192.168.3.52:9355 - *********************
RMMSTATS_ EventAnnouncer nRot: 646
RMMSTATS_ Unicast Pending #: 0
RMMSTATS_ UnicastPTransmitter status:
RMMSTATS_Fireout: p3 r12990. HBSender: ccp187 snp59. TimingThrd: p1 r5885. ConnPending: r224 p0 np0
RMMSTATS_------------ QueueT: ibmwas1Cell01\ibmwas2Node01\member2#DCS#PTP#CON#DefaultCoreGroup#PtpRmmNode|ibmwas1Cell01\ibmwas2Node01\member2|192.168.3.52|9355. Stream Id: -2189944965944826741
RMMSTATS_ Messages: 44. Front: 44. Pending: 0. Quarantine: false
RMMSTATS_------------ QueueT: Nameless (ByteArray) Queue[B@a980a98. Stream Id: -2115917047069924213
_RMM_STATS_ Messages: 15. Front: 15. Pending: 0. Quarantine: false
_RMM_STATS_------------ QueueT: Nameless (ByteArray) Queue[B@72927292. Stream Id: -2116198522046634869
_RMM_STATS_ Messages: 0. Front: 0. Pending: 0. Quarantine: false
_RMM_STATS_------------ QueueT: Nameless (ByteArray) Queue[B@2bb82bb8. Stream Id: -2191070865851669365
_RMM_STATS_ Messages: 1. Front: 1. Pending: 0. Quarantine: false
_RMM_STATS_------------ QueueT: Nameless (ByteArray) Queue[B@2320232. Stream Id: -2190226440921537397
_RMM_STATS_ Messages: 2799. Front: 2799. Pending: 0. Quarantine: false
_RMM_STATS_********************************
*********************** DiscoverySubServer::dump() *******************
Stack name: DefaultCoreGroup
Defined:
{ibmwas1Cell01\hrcs9Node\member1=PtpRmmNode|ibmwas1Cell01\hrcs9Node\member1|UnicastConnection@1746429976, address: 127.0.0.1:9355 | 2020, isOutbound: true, isValid: true (false false).|11, ibmwas1Cell01\hrcs9Node\nodeagent=PtpRmmNode|ibmwas1Cell01\hrcs9Node\nodeagent|null|11, ibmwas1Cell01\ibmwas2Node01\nodeagent=PtpRmmNode|ibmwas1Cell01\ibmwas2Node01\nodeagent|UnicastConnection@1819962490, address: 192.168.3.52:9353 | 2854, isOutbound: true, isValid: true (false false).|21, ibmwas1Cell01\hrcs9Node\server1=PtpRmmNode|ibmwas1Cell01\hrcs9Node\server1|UnicastConnection@529538960, address: 127.0.0.1:9353 | 2855, isOutbound: true, isValid: true (false false).|11, ibmwas1Cell01\ibmwas1CellManager01\dmgr=PtpRmmNode|ibmwas1Cell01\ibmwas1CellManager01\dmgr|UnicastConnection@1635541372, address: 192.168.3.51:9352 | 2017, isOutbound: true, isValid: true (false false).|11}
+Data for directive [defaultdcs] obtained. =
==> Dump complete for com.ibm.ws.dcs.vri.common.util.DCSDiagnosticModule = Thu Nov 06 00:43:51 PKT 2008