Problem establishing new feeder channels following an election when ReplicationGroupAdmin or DbPing used
This question pertains to 5.0.84. I didn't find any prior posts here with specifically the same problem.
My application uses JE HA with replication across 4 nodes. We've been testing various failure scenarios in which we network-isolate the Master node. We see the 3 surviving replicas call an election, a Master is determined, feeder channels are established and the overall replication group resumes its expected functionality. We also tested scenarios in which the Master of the now-reduced 3-node replication group is similarly network-isolated. The results generally worked but were less consistent when the group transitions from 3 to 2 nodes. We thought that adjusting the quorum (electable group size override) was the proper way to address this issue and had good results by manually adjusting the electable group size override downward.
We implemented a StateChangeListener and in its stateChange() method are attempting to automate adjustment of the electable group size override by calling getNodeState() methods of (alternately) DbPing and ReplicationGroupAdmin to query each of the possible 4 nodes to see which were reachable and in a non-DETACHED state. However, it appears that the very use of either DbPing or ReplicationGroupAdmin within our application at any node causes the establishment of feeder channels following an election of a new Master to fail (regardless the number of surviving electable nodes). This seems rather counterintuitive, or that introducing either DbPing or ReplicationGroupAdmin within our application code introduces a Heisenberg uncertainty principle-effect in which attempting to observe the number of active nodes causes the behavior of the replication group, or more specifically--the feeder channel establishment--to change. Is there a prohibition against using either DbPing, ReplicationGroupAdmin, or the getNodeState() methods within an electable node application? Likewise do using these objects/methods within a monitor app cause a change in behavior among electable nodes following an election? We seem to be observing such behavior.
Thanks,
Ted
Here is the je.info.0 log, from node01. Node02 initially is Master. At 17:57:55, we network-isolate node02. We see an election is called, node01 wins and begins trying to establish feeders but cannot.
130903 17:55:56:752 INFO [node01] Started ServiceDispatcher. HostPort=ap01.domain.acme.net:13000
130903 17:55:56:867 INFO [node01] Current group size: 4
130903 17:55:56:868 INFO [node01] Existing node node01 querying for a current master.
130903 17:55:56:913 INFO [node01] Node node01 started
130903 17:55:56:914 INFO [node01] Election initiated; election #1
130903 17:55:56:926 INFO [node01] Started election thread Tue Sep 03 17:55:56 PDT 2013
130903 17:56:29:308 INFO [node01] Master changed to node02
130903 17:56:29:310 INFO [node01] Election finished. Elapsed time: 32396ms
130903 17:56:29:310 INFO [node01] Exiting election after 5 retries
130903 17:56:29:311 INFO [node01] Election thread exited. Group master: node02(64)
130903 17:56:29:310 INFO [node01] Replica loop started with master: node02(64)
130903 17:56:29:343 INFO [node01] Replica-feeder handshake start
130903 17:56:29:536 INFO [node01] Replica-feeder node02 handshake completed.
130903 17:56:29:553 INFO [node01] Replica-feeder node02 syncup started. Replica range: first=843,326 last=843,393 sync=843,393 txnEnd=843,393
130903 17:56:29:650 INFO [node01] Rollback to matchpoint 843,393 at 0x33/0x2d38 status=No active txns, nothing to rollback
130903 17:56:29:651 INFO [node01] Replica-feeder node02 start stream at VLSN: 843,394
130903 17:56:29:652 INFO [node01] Replica-feeder node02 syncup ended. Elapsed time: 102ms
130903 17:56:29:658 INFO [node01] Replica initialization completed. Replica VLSN: 843,393 Heartbeat master commit VLSN: 843,393 VLSN delta: 0
130903 17:56:29:662 INFO [node01] Joined group as a replica. join consistencyPolicy=PointConsistencyPolicy targetVLSN=843,393 first=843,326 last=843,393 sync=843,393 txnEnd=843,393
130903 17:56:29:667 INFO [node01] Replay thread started. Message queue size:1000
130903 17:56:29:666 INFO [node01] Refreshed 0 monitors.
130903 17:56:30:939 WARNING [node01] Electable group size override changed to:3
130903 17:57:54:860 INFO [node01] Inactive channel: node02(64) forced close. Timeout: 7000ms.
130903 17:57:55:406 INFO [node01] Exiting inner Replica loop.
130903 17:57:55:407 INFO [node01] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:57:56:049 INFO [node01] Election initiated; election #2
130903 17:57:56:050 INFO [node01] Election in progress. Waiting....
130903 17:57:56:057 INFO [node01] Started election thread Tue Sep 03 17:57:56 PDT 2013
130903 17:58:08:115 INFO [node01] Master changed to node01
130903 17:58:08:116 INFO [node01] Election finished. Elapsed time: 12067ms
130903 17:58:08:121 INFO [node01] Request for unknown Service: Feeder Registered services: [Acceptor, Learner, LogFileFeeder, LDiff, NodeState, Group, BinaryNodeState]
130903 17:58:08:122 INFO [node01] Request for unknown Service: Feeder Registered services: [Acceptor, Learner, LogFileFeeder, LDiff, NodeState, Group, BinaryNodeState]
130903 17:58:08:122 INFO [node01] Exiting election after 2 retries
130903 17:58:08:123 INFO [node01] Election thread exited. Group master: node01(61)
130903 17:58:09:124 INFO [node01] Request for unknown Service: Feeder Registered services: [Acceptor, Learner, LogFileFeeder, LDiff, NodeState, Group, BinaryNodeState]
130903 17:58:09:130 INFO [node01] Request for unknown Service: Feeder Registered services: [Acceptor, Learner, LogFileFeeder, LDiff, NodeState, Group, BinaryNodeState]
130903 17:58:10:128 INFO [node01] Request for unknown Service: Feeder Registered services: [Acceptor, Learner, LogFileFeeder, LDiff, NodeState, Group, BinaryNodeState]
130903 17:58:10:137 INFO [node01] Request for unknown Service: Feeder Registered services: [Acceptor, Learner, LogFileFeeder, LDiff, NodeState, Group, BinaryNodeState]
130903 17:58:11:132 INFO [node01] Request for unknown Service: Feeder Registered services: [Acceptor, Learner, LogFileFeeder, LDiff, NodeState, Group, BinaryNodeState]
... The same message repeats ad nauseum....
Here is node02's je.info.0 log for the same time period. Node02 starts as Master but this host becomes network-isolated and shuts down its feeder channels at 17:57:55:
130903 17:56:26:861 INFO [node02] Chose lowest utilized file for cleaning. fileChosen: 0x2f totalUtilization: 50 bestFileUtilization: 0 lnSizeCorrectionFactor: 0.55577517 isProbe: false
130903 17:56:26:912 INFO [node02] CleanerRun 1 ends on file 0x2f probe=false invokedFromDaemon=true finished=true fileDeleted=false nEntriesRead=1 nINsObsolete=0 nINsCleaned=0 nINsDead=0 nINsMigrated=0 nBINDeltasObsolete=0 nBINDeltasCleaned=0 nBINDeltasDead=0 nBINDeltasMigrated=0 nLNsObsolete=0 nLNsCleaned=0 nLNsDead=0 nLNsMigrated=0 nLNsMarked=0 nLNQueueHits=0 nLNsLocked=0 logSummary=<CleanerLogSummary endFileNumAtLastAdjustment="0x33" initialAdjustments="5" recentLNSizesAndCounts="Cor:1274201/3219-Est:1494628/3219 Cor:535/6-Est:965/11 Cor:788/8-Est:1190/13 Cor:789/8-Est:1289/14 Cor:714/2-Est:1310/9 Cor:37694/611-Est:178596/619 Cor:168637/3961-Est:1164331/3961 Cor:62/1-Est:2286/21 Cor:22130/1446-Est:417561/1451 Cor:2429020/9915-Est:3837944/9915 "> inSummary=<INSummary totalINCount="0" totalINSize="0" totalBINDeltaCount="0" totalBINDeltaSize="0" obsoleteINCount="0" obsoleteINSize="0" obsoleteBINDeltaCount="0" obsoleteBINDeltaSize="0"/> estFileSummary=<summary totalCount="1" totalSize="31" totalINCount="0" totalINSize="0" totalLNCount="0" totalLNSize="0" maxLNSize="0" obsoleteINCount="0" obsoleteLNCount="0" obsoleteLNSize="0" obsoleteLNSizeCounted="0" getObsoleteSize="31" getObsoleteINSize="0" getObsoleteLNSize="0" getMaxObsoleteSize="31" getMaxObsoleteLNSize="0" getAvgObsoleteLNSizeNotCounted="NaN"/> recalcFileSummary=<summary totalCount="1" totalSize="31" totalINCount="0" totalINSize="0" totalLNCount="0" totalLNSize="0" maxLNSize="0" obsoleteINCount="0" obsoleteLNCount="0" obsoleteLNSize="0" obsoleteLNSizeCounted="0" getObsoleteSize="31" getObsoleteINSize="0" getObsoleteLNSize="0" getMaxObsoleteSize="31" getMaxObsoleteLNSize="0" getAvgObsoleteLNSizeNotCounted="NaN"/> lnSizeCorrection=0.55577517 newLnSizeCorrection=0.55577517 estimatedUtilization=0 correctedUtilization=0 recalcUtilization=0
130903 17:56:26:973 INFO [node02] Started ServiceDispatcher. HostPort=ap02.domain.acme.net:13000
130903 17:56:26:992 INFO [node02] Request for unknown Service: Acceptor Registered services: []
130903 17:56:27:108 INFO [node02] Current group size: 4
130903 17:56:27:108 INFO [node02] Existing node node02 querying for a current master.
130903 17:56:27:186 INFO [node02] Node node02 started
130903 17:56:27:186 INFO [node02] Election initiated; election #1
130903 17:56:27:206 INFO [node02] Started election thread Tue Sep 03 17:56:27 PDT 2013
130903 17:56:29:290 INFO [node02] Winning proposal: Proposal(00000140e6787717:0000000000000000000000000a000253:00000001) Value: Value:10.0.2.83$$$13000$$$node02$$$64
130903 17:56:29:299 INFO [node02] Master changed to node02
130903 17:56:29:301 INFO [node02] Election finished. Elapsed time: 2114ms
130903 17:56:29:302 INFO [node02] Feeder manager accepting requests.
130903 17:56:29:309 INFO [node02] Request for unknown Service: Feeder Registered services: [Acceptor, Learner, LogFileFeeder, LDiff, NodeState, Group, BinaryNodeState]
130903 17:56:29:312 INFO [node02] Joining group as master
130903 17:56:29:317 INFO [node02] Refreshed 0 monitors.
130903 17:56:29:318 INFO [node02] Election thread exited. Group master: node02(64)
130903 17:56:29:374 INFO [node02] Feeder accepted connection from java.nio.channels.SocketChannel[connected local=/10.0.2.83:13000 remote=/10.0.2.82:60052]
130903 17:56:29:477 INFO [node02] Feeder-replica handshake start
130903 17:56:29:539 INFO [node02] Feeder-replica node01 handshake completed.
130903 17:56:29:542 INFO [node02] Feeder-replica node01 syncup started. Feeder range: first=843,326 last=843,393 sync=843,393 txnEnd=843,393
130903 17:56:29:654 INFO [node02] Feeder-replica node01 start stream at VLSN: 843,394
130903 17:56:29:656 INFO [node02] Feeder-replica node01 syncup ended. Elapsed time: 113ms
130903 17:56:29:660 INFO [node02] Feeder output thread for replica node01 started at VLSN 843,394 master at 843,393 VLSN delta=-1 socket=(node01(61))java.nio.channels.SocketChannel[connected local=/10.0.2.83:13000 remote=/10.0.2.82:60052]
130903 17:56:30:321 INFO [node02] Feeder accepted connection from java.nio.channels.SocketChannel[connected local=/10.0.2.83:13000 remote=/10.0.2.63:47920]
130903 17:56:30:326 INFO [node02] Feeder-replica handshake start
130903 17:56:30:349 INFO [node02] Feeder-replica node04 handshake completed.
130903 17:56:30:350 INFO [node02] Feeder-replica node04 syncup started. Feeder range: first=843,326 last=843,395 sync=843,395 txnEnd=843,395
130903 17:56:30:363 INFO [node02] Feeder-replica node04 start stream at VLSN: 843,394
130903 17:56:30:364 INFO [node02] Feeder-replica node04 syncup ended. Elapsed time: 14ms
130903 17:56:30:383 INFO [node02] Feeder output thread for replica node04 started at VLSN 843,394 master at 843,395 VLSN delta=1 socket=(node04(62))java.nio.channels.SocketChannel[connected local=/10.0.2.83:13000 remote=/10.0.2.63:47920]
130903 17:56:31:678 WARNING [node02] Electable group size override changed to:3
130903 17:57:21:200 INFO [node02] Feeder accepted connection from java.nio.channels.SocketChannel[connected local=/10.0.2.83:13000 remote=/10.0.2.84:38224]
130903 17:57:21:206 INFO [node02] Feeder-replica handshake start
130903 17:57:21:272 INFO [node02] Feeder-replica node03 handshake completed.
130903 17:57:21:273 INFO [node02] Feeder-replica node03 syncup started. Feeder range: first=843,326 last=843,399 sync=843,399 txnEnd=843,399
130903 17:57:21:389 INFO [node02] Feeder-replica node03 start stream at VLSN: 843,394
130903 17:57:21:390 INFO [node02] Feeder-replica node03 syncup ended. Elapsed time: 117ms
130903 17:57:21:392 INFO [node02] Feeder output thread for replica node03 started at VLSN 843,394 master at 843,399 VLSN delta=5 socket=(node03(63))java.nio.channels.SocketChannel[connected local=/10.0.2.83:13000 remote=/10.0.2.84:38224]
130903 17:57:55:115 INFO [node02] Inactive channel: node01(61) forced close. Timeout: 7000ms.
130903 17:57:55:115 INFO [node02] Inactive channel: node04(62) forced close. Timeout: 7000ms.
130903 17:57:55:116 INFO [node02] Shutting down feeder for replica node01 Reason: null write time: 2ms Avg write time: 98us
130903 17:57:55:125 INFO [node02] Inactive channel: node03(63) forced close. Timeout: 7000ms.
130903 17:57:55:126 INFO [node02] Shutting down feeder for replica node04 Reason: null write time: 3ms Avg write time: 147us
130903 17:57:55:127 INFO [node02] Shutting down feeder for replica node03 Reason: null write time: 1ms Avg write time: 90us
130903 17:57:55:409 INFO [node02] Feeder output for replica node01 shutdown. feeder VLSN: 843,402 currentTxnEndVLSN: 843,401
130903 17:57:55:410 INFO [node02] Feeder output for replica node03 shutdown. feeder VLSN: 843,402 currentTxnEndVLSN: 843,401
130903 17:57:55:411 INFO [node02] Feeder output for replica node04 shutdown. feeder VLSN: 843,402 currentTxnEndVLSN: 843,401
130903 17:58:49:456 INFO [node02] Master changed to node01
130903 17:58:50:227 INFO [node02] Master change: Master change. Node master id: node02(64) Group master id: node01(61)
130903 17:58:50:228 INFO [node02] Releasing commit block latch
130903 17:58:50:229 INFO [node02] Feeder manager exited. CurrentTxnEnd VLSN: 843,401
130903 17:58:50:230 INFO [node02] RepNode main thread shutting down.
130903 17:58:50:233 INFO [node02] RepNode shutdown exception:
(JE 5.0.84) node02(64):/data/node02 com.sleepycat.je.rep.stream.MasterStatus$MasterSyncException: Master change. Node master id: node02(64) Group master id: node01(61) MASTER_TO_REPLICA_TRANSITION: This node was a master and must reinitialize internal state to become a replica. The application must close and reopen all Environment handles. Environment is invalid and must be closed.node02(64)[MASTER]
No feeders.
GlobalCBVLSN=843,393
Group info [RepGroup] 8e8b97b5-f9bc-4b9e-bb6e-56edfbe79674
Representation version: 2
Change version: 4
Max rep node ID: 64
Node:node02 ap02.domain.acme.net:13000 (is member) changeVersion:4 LocalCBVLSN:843,393 at:Tue Sep 03 17:56:31 PDT 2013
Node:node03 ap03.domain.acme.net:13000 (is member) changeVersion:3 LocalCBVLSN:843,393 at:Tue Sep 03 17:57:21 PDT 2013
Node:node01 ap01.domain.acme.net:13000 (is member) changeVersion:1 LocalCBVLSN:843,393 at:Tue Sep 03 17:56:29 PDT 2013
Node:node04 ap04.domain.acme.net:13000 (is member) changeVersion:2 LocalCBVLSN:843,393 at:Tue Sep 03 17:56:30 PDT 2013
vlsnRange=first=843,326 last=843,401 sync=843,401 txnEnd=843,401
lastReplayedTxn=null lastReplayedVLSN=843,393 numActiveReplayTxns=0
130903 17:58:50:234 INFO [node02] Shutting down node node02(64)
130903 17:58:50:235 INFO [node02] Elections shutdown initiated
130903 17:58:50:238 INFO [node02] Elections shutdown completed
130903 17:58:50:239 INFO [node02] RepNode main thread: MASTER node02(64) exited.
130903 17:58:50:240 INFO [node02] ServiceDispatcher shutdown starting. HostPort=ap02.domain.acme.net:13000 Registered services: []
130903 17:58:50:243 INFO [node02] ServiceDispatcher shutdown completed. HostPort=ap02.domain.acme.net:13000
130903 17:58:50:243 INFO [node02] node02(64) shutdown completed.
Here is node03's log. We see that node01 wins the election but
130903 17:57:20:961 INFO [node03] CleanerRun 1 ends on file 0x2f probe=false invokedFromDaemon=true finished=true fileDeleted=false nEntriesRead=1 nINsObsolete=0 nINsCleaned=0 nINsDead=0 nINsMigrated=0 nBINDeltasObsolete=0 nBINDeltasCleaned=0 nBINDeltasDead=0 nBINDeltasMigrated=0 nLNsObsolete=0 nLNsCleaned=0 nLNsDead=0 nLNsMigrated=0 nLNsMarked=0 nLNQueueHits=0 nLNsLocked=0 logSummary=<CleanerLogSummary endFileNumAtLastAdjustment="0x32" initialAdjustments="5" recentLNSizesAndCounts="Cor:1274201/3219-Est:1494628/3219 Cor:535/6-Est:965/11 Cor:788/8-Est:1190/13 Cor:789/8-Est:1289/14 Cor:714/2-Est:1310/9 Cor:37694/611-Est:178596/619 Cor:168637/3961-Est:1164331/3961 Cor:62/1-Est:2286/21 Cor:22130/1446-Est:417561/1451 Cor:2429020/9915-Est:3837944/9915 "> inSummary=<INSummary totalINCount="0" totalINSize="0" totalBINDeltaCount="0" totalBINDeltaSize="0" obsoleteINCount="0" obsoleteINSize="0" obsoleteBINDeltaCount="0" obsoleteBINDeltaSize="0"/> estFileSummary=<summary totalCount="1" totalSize="31" totalINCount="0" totalINSize="0" totalLNCount="0" totalLNSize="0" maxLNSize="0" obsoleteINCount="0" obsoleteLNCount="0" obsoleteLNSize="0" obsoleteLNSizeCounted="0" getObsoleteSize="31" getObsoleteINSize="0" getObsoleteLNSize="0" getMaxObsoleteSize="31" getMaxObsoleteLNSize="0" getAvgObsoleteLNSizeNotCounted="NaN"/> recalcFileSummary=<summary totalCount="1" totalSize="31" totalINCount="0" totalINSize="0" totalLNCount="0" totalLNSize="0" maxLNSize="0" obsoleteINCount="0" obsoleteLNCount="0" obsoleteLNSize="0" obsoleteLNSizeCounted="0" getObsoleteSize="31" getObsoleteINSize="0" getObsoleteLNSize="0" getMaxObsoleteSize="31" getMaxObsoleteLNSize="0" getAvgObsoleteLNSizeNotCounted="NaN"/> lnSizeCorrection=0.55577517 newLnSizeCorrection=0.55577517 estimatedUtilization=0 correctedUtilization=0 recalcUtilization=0
130903 17:57:20:991 INFO [node03] Started ServiceDispatcher. HostPort=ap03.domain.acme.net:13000
130903 17:57:21:111 INFO [node03] Current group size: 4
130903 17:57:21:112 INFO [node03] Existing node node03 querying for a current master.
130903 17:57:21:155 INFO [node03] Master changed to node02
130903 17:57:21:182 INFO [node03] Node node03 started
130903 17:57:21:183 INFO [node03] Replica loop started with master: node02(64)
130903 17:57:21:205 INFO [node03] Replica-feeder handshake start
130903 17:57:21:269 INFO [node03] Replica-feeder node02 handshake completed.
130903 17:57:21:282 INFO [node03] Replica-feeder node02 syncup started. Replica range: first=843,326 last=843,393 sync=843,393 txnEnd=843,393
130903 17:57:21:385 INFO [node03] Rollback to matchpoint 843,393 at 0x32/0x55da status=No active txns, nothing to rollback
130903 17:57:21:386 INFO [node03] Replica-feeder node02 start stream at VLSN: 843,394
130903 17:57:21:387 INFO [node03] Replica-feeder node02 syncup ended. Elapsed time: 105ms
130903 17:57:21:391 INFO [node03] Replica initialization completed. Replica VLSN: 843,393 Heartbeat master commit VLSN: 843,399 VLSN delta: 6
130903 17:57:21:396 INFO [node03] Replay thread started. Message queue size:1000
130903 17:57:21:417 INFO [node03] Joined group as a replica. join consistencyPolicy=PointConsistencyPolicy targetVLSN=843,399 first=843,326 last=843,399 sync=843,399 txnEnd=843,399
130903 17:57:21:421 INFO [node03] Refreshed 0 monitors.
130903 17:57:55:069 INFO [node03] Inactive channel: node02(64) forced close. Timeout: 7000ms.
130903 17:57:55:406 INFO [node03] Exiting inner Replica loop.
130903 17:57:55:407 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:57:56:059 INFO [node03] Election initiated; election #1
130903 17:57:56:068 INFO [node03] Started election thread Tue Sep 03 17:57:56 PDT 2013
130903 17:58:08:101 INFO [node03] Winning proposal: Proposal(00000140e679d235:0000000000000000000000000a000254:00000001) Value: Value:10.0.2.82$$$13000$$$node01$$$61
130903 17:58:08:108 INFO [node03] Master changed to node01
130903 17:58:08:112 INFO [node03] Election finished. Elapsed time: 12053ms
130903 17:58:08:113 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:08:124 INFO [node03] Exiting inner Replica loop.
130903 17:58:08:125 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:08:126 INFO [node03] Retry #: 0/10 Will retry replica loop after 1000ms.
130903 17:58:09:127 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:09:131 INFO [node03] Exiting inner Replica loop.
130903 17:58:09:131 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:09:132 INFO [node03] Retry #: 1/10 Will retry replica loop after 1000ms.
130903 17:58:10:133 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:10:138 INFO [node03] Exiting inner Replica loop.
130903 17:58:10:138 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:10:139 INFO [node03] Retry #: 2/10 Will retry replica loop after 1000ms.
130903 17:58:11:139 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:11:142 INFO [node03] Exiting inner Replica loop.
130903 17:58:11:142 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:11:143 INFO [node03] Retry #: 3/10 Will retry replica loop after 1000ms.
130903 17:58:12:144 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:12:146 INFO [node03] Exiting inner Replica loop.
130903 17:58:12:147 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:12:147 INFO [node03] Retry #: 4/10 Will retry replica loop after 1000ms.
130903 17:58:13:148 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:13:162 INFO [node03] Exiting inner Replica loop.
130903 17:58:13:222 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:13:222 INFO [node03] Retry #: 5/10 Will retry replica loop after 1000ms.
130903 17:58:14:223 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:14:225 INFO [node03] Exiting inner Replica loop.
130903 17:58:14:226 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:14:226 INFO [node03] Retry #: 6/10 Will retry replica loop after 1000ms.
130903 17:58:15:227 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:15:229 INFO [node03] Exiting inner Replica loop.
130903 17:58:15:230 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:15:230 INFO [node03] Retry #: 7/10 Will retry replica loop after 1000ms.
130903 17:58:16:231 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:16:233 INFO [node03] Exiting inner Replica loop.
130903 17:58:16:234 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:16:234 INFO [node03] Retry #: 8/10 Will retry replica loop after 1000ms.
130903 17:58:17:235 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:17:237 INFO [node03] Exiting inner Replica loop.
130903 17:58:17:238 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:17:238 INFO [node03] Retry #: 9/10 Will retry replica loop after 1000ms.
130903 17:58:18:112 INFO [node03] Election thread exited. Group master: node01(61)
130903 17:58:18:239 INFO [node03] Replica loop started with master: node01(61)
130903 17:58:18:280 INFO [node03] Exiting inner Replica loop.
130903 17:58:18:280 INFO [node03] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:18:281 INFO [node03] Failed to recover from exception: Failed after retries: 10 with retry interval: 1000ms., despite 10 retries.
com.sleepycat.je.rep.impl.node.Replica$ConnectRetryException: Failed after retries: 10 with retry interval: 1000ms.
at com.sleepycat.je.rep.impl.node.Replica.createReplicaFeederChannel(Replica.java:777)
at com.sleepycat.je.rep.impl.node.Replica.initReplicaLoop(Replica.java:578)
at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:392)
at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:328)
at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:1402)
130903 17:58:18:282 INFO [node03] Election initiated; election #2
130903 17:58:18:282 INFO [node03] Election in progress. Waiting....
.... The retries and the ConnectRetryException repeat ad nauseum....
We see a similar pattern in the log of node04. We observe at 17:58:08:101 that node01 has become Master but we're unable to establish a feeder connection to it:
130903 17:55:48:149 INFO [node04] Started ServiceDispatcher. HostPort=ap04.domain.acme.net:13000
130903 17:55:48:230 INFO [node04] Current group size: 4
130903 17:55:48:231 INFO [node04 Existing node node04 querying for a current master.
130903 17:55:48:258 INFO [node04] Node node04 started
130903 17:55:48:259 INFO [node04] Election initiated; election #1
130903 17:55:48:266 INFO [node04] Started election thread Tue Sep 03 17:55:48 PDT 2013130903 17:56:29:300 INFO [node04] Master changed to node02
130903 17:56:29:301 INFO [node04] Election finished. Elapsed time: 41043ms
130903 17:56:29:301 INFO [node04] Replica loop started with master: node02(64)130903 17:56:29:303 INFO [node04] Exiting election after 5 retries130903 17:56:29:304 INFO [node04] Election thread exited. Group master: node02(6
4)
130903 17:56:29:309 INFO [node04] Exiting inner Replica loop.
130903 17:56:29:310 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:56:29:310 INFO [node04] Retry #: 0/10 Will retry replica loop after 10
00ms.130903 17:56:30:310 INFO [node04] Replica loop started with master: node02(64)
130903 17:56:30:324 INFO [node04] Replica-feeder handshake start
130903 17:56:30:348 INFO [node04] Replica-feeder node02 handshake completed.
130903 17:56:30:352 INFO [node04] Replica-feeder node02 syncup started. Replica range: first=843,326 last=843,393 sync=843,393 txnEnd=843,393
130903 17:56:30:361 INFO [node04] Rollback to matchpoint 843,393 at 0x34/0x1628
status=No active txns, nothing to rollback
130903 17:56:30:362 INFO [node04] Replica-feeder node02 start stream at VLSN: 843,394
130903 17:56:30:362 INFO [node04] Replica-feeder node02 syncup ended. Elapsed time: 10ms
130903 17:56:30:382 INFO [node04] Replica initialization completed. Replica VLSN: 843,393 Heartbeat master commit VLSN: 843,395 VLSN delta: 2
130903 17:56:30:386 INFO [node04] Replay thread started. Message queue size:1000
130903 17:56:30:397 INFO [node04] Joined group as a replica. join consistencyPolicy=PointConsistencyPolicy targetVLSN=843,395 first=843,326 last=843,395 sync=843,395 txnEnd=843,395
130903 17:56:30:401 INFO [node04] Refreshed 0 monitors.
130903 17:56:30:922 WARNING [node04] Electable group size override changed to:3
130903 17:57:55:223 INFO [node04] Inactive channel: node02(64) forced close. Timeout: 7000ms.
130903 17:57:55:398 INFO [node04] Exiting inner Replica loop.
130903 17:57:55:399 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:57:55:401 INFO [node04] Election initiated; election #2
130903 17:57:55:401 INFO [node04] Election in progress. Waiting....
130903 17:57:55:406 INFO [node04] Started election thread Tue Sep 03 17:57:55 PDT 2013
130903 17:58:08:101 INFO [node04] Master changed to node01
130903 17:58:08:102 INFO [node04] Election finished. Elapsed time: 12701ms
130903 17:58:08:102 INFO [node04] Exiting election after 2 retries
130903 17:58:08:102 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:08:103 INFO [node04] Election thread exited. Group master: node01(61)
130903 17:58:08:110 INFO [node04] Exiting inner Replica loop.
130903 17:58:08:110 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:08:111 INFO [node04] Retry #: 0/10 Will retry replica loop after 1000ms.
130903 17:58:09:111 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:09:114 INFO [node04] Exiting inner Replica loop.
130903 17:58:09:114 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:09:114 INFO [node04] Retry #: 1/10 Will retry replica loop after 1000ms.
130903 17:58:10:115 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:10:117 INFO [node04] Exiting inner Replica loop.
130903 17:58:10:118 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:10:118 INFO [node04] Retry #: 2/10 Will retry replica loop after 1000ms.
130903 17:58:11:118 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:11:121 INFO [node04] Exiting inner Replica loop.
130903 17:58:11:121 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:11:122 INFO [node04] Retry #: 3/10 Will retry replica loop after 1000ms.
130903 17:58:12:122 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:12:125 INFO [node04] Exiting inner Replica loop.
130903 17:58:12:125 INFO [node04] Retry #: 4/10 Will retry replica loop after 1000ms.
130903 17:58:13:126 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:13:129 INFO [node04] Exiting inner Replica loop.
130903 17:58:13:129 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:13:129 INFO [node04] Retry #: 5/10 Will retry replica loop after 1000ms.
130903 17:58:14:130 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:14:132 INFO [node04] Exiting inner Replica loop.
130903 17:58:14:133 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:14:133 INFO [node04] Retry #: 6/10 Will retry replica loop after 1000ms.
130903 17:58:15:133 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:15:136 INFO [node04] Exiting inner Replica loop.
130903 17:58:15:136 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:15:137 INFO [node04] Retry #: 7/10 Will retry replica loop after 1000ms.
130903 17:58:16:137 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:16:140 INFO [node04] Exiting inner Replica loop.
130903 17:58:16:140 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:16:140 INFO [node04] Retry #: 8/10 Will retry replica loop after 1000ms.
130903 17:58:17:141 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:17:143 INFO [node04] Exiting inner Replica loop.
130903 17:58:17:144 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:17:144 INFO [node04] Retry #: 9/10 Will retry replica loop after 1000ms.
130903 17:58:18:145 INFO [node04] Replica loop started with master: node01(61)
130903 17:58:18:147 INFO [node04] Exiting inner Replica loop.
130903 17:58:18:148 INFO [node04] Replica stats - Lag waits: 0 Lag wait time: 0ms. VLSN waits: 0 Lag wait time: 0ms.
130903 17:58:18:148 INFO [node04] Failed to recover from exception: Failed after retries: 10 with retry interval: 1000ms., despite 10 retries.
com.sleepycat.je.rep.impl.node.Replica$ConnectRetryException: Failed after retries: 10 with retry interval: 1000ms.
at com.sleepycat.je.rep.impl.node.Replica.createReplicaFeederChannel(Replica.java:777)
at com.sleepycat.je.rep.impl.node.Replica.initReplicaLoop(Replica.java:578)
at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:392)
at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:328)
at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:1402)
130903 17:58:18:149 INFO [node04] Election initiated; election #3
130903 17:58:18:149 INFO [node04] Election in progress. Waiting....
130903 17:58:18:150 INFO [node04] Started election thread Tue Sep 03 17:58:18 PDT 2013
....The retries and the ConnectRetryException repeat ad nauseum....
The issue is that you're hitting an unsupported model by violating the contract of the StateChangeListener by issuing an expensive operation in the middle of the stateChange method. From http://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/rep/StateChangeListener.html#stateChange(com.sleepycat.je.rep.StateChangeEvent), we caution:
void stateChange(StateChangeEvent stateChangeEvent) throws RuntimeException ....
This method should do the minimal amount of work, queuing any resource intensive operations for processing by another thread before returning to the caller, so that it does not unduly delay the other housekeeping operations performed by the internal thread which invokes this method.
In other words, we really mean that the work done by stateChange should not issue any I/O or network communication, and if it does so, it should let this thread return, and do the expensive work in another thread.
This seems rather counterintuitive, or that introducing either DbPing or ReplicationGroupAdmin within our application code introduces a Heisenberg uncertainty principle-effect in which attempting to observe the number of active nodes causes the behavior of the replication group, or more specifically--the feeder channel establishment--to change. Is there a prohibition against using either DbPing, ReplicationGroupAdmin, or the getNodeState() methods within an electable node application? Likewise do using these objects/methods within a monitor app cause a change in behavior among electable nodes following an election? We seem to be observing such behavior.
While an application can certainly use DbPing or ReplicationGroupAdmin, the problem is that the state change listener is executing within the critical paths of the replication group state changes. You mentioned monitors, and an alternate model of watching and managing replication group membership is to use http://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/rep/monitor/Monitor.html. This class, and the associated MonitorStateChangeListener, run outside the ReplicatedEnvironment, and could be another way of implementing application logic.
So in summary, you should either use com.sleepycat.je.rep.monitor.Monitor* to implement your logic, or have the implementation of StateChangeListener provoke another thread to do the work asynchronously, so that stateChange() can return quickly.
Please let me know if this helps.
Thanks,
Bogdan
Similar Messages
-
Plz fix the problem in newfeed wont,t get new feed Plz fix it
Hi,
NEVER delete the files manually. As a admins we have to follow proper Uninstall and reinstall procedures as above one gentle man said.
Here we will definately have some extra files also, like sapreorg, and backup file manually created if . Those also should be uninstalled properly.
Also, we should maintain one back up copy of our file systems in each servers.
just refer my system file structure, and proceed further.
http://img158.imageshack.us/img158/591/bdfll0.jpg
bye... -
Hi all Im having a problem viewing the new HD channels. I have subscribed to entertainment extra and have the HD package. I can watch streamed HD movies etc no problem. I can view comedy central, history etc ok however I cant watch the HD version of these channels. I spent 2 hours last night with a BT rep trying to get the channels to work but was left with the line was being reset and would take an hour.
I just tried again today and still not working.
I have tried...
switching off both home hub 4 and the vision box
connected the vision box directly to the home hub 4
resetting my subscription
resyncing with BT servers
checked the test channel 900
still not working! Can anyone shed some light on this please?
I have BT Vision + with Entertainment extra and free bt sports (I signed up in June 13)Check the following.
Help & Settings > Settings > Entitlement Information
Here are possible entitlements mapped to channels.
101 - 104 : PSB HD
407 - 433 : Essential Extra / Unlimited Extra
446 - 453 : HD Extra
501 : Sky Sports 1
502 : Sky Sports 2
507 - 509 : BT Sport SD
510 - 512 : BT Sport HD
513 - 514 : Essential Extra / Unlimited Extra
530 - 540 : Sky Movies
555 - 563 : Kids Extra
You need 'HD Extra' for channels 446 - 453. I have the free 'BT Sport HD' but that does not cover entertainment HD channels. You need to subscribe to the HD Extra bolt-on for £3 for them. Have you ordered HD Extra (it only became available from last Sunday)? -
Problem with new ITune 7.01, Itunes freezes when I connect my nano!
I just installed the latest Itune software update and my nano update, after the upgrade Itune when I connected the nano, it said it was corrupted and asked me if I wanted to restore it. I said yes, now everytime I try to connect my nano via a USB cable, Itunes freezes. How do I fix this issue? I am wondering if I need to have the lastest version of the nano for version 7.01 of Itunes to work, yet everytime I try to connect my nano, Itunes freezes?! Can anyone help me?
duplicate thread of https://discussions.apple.com/thread/3371308
its sure a bug and we hope they read feedback reports and forum reports from us users (better known as beta testers)
iphone 4 cant connect to itunes 10.5 on lion, reported many times -
How to format new hard drive for Apple TV when you can't use old HD
I need to know how to format a new HD to install in my Apple TV where I can't use the old HD to clone or otherwise create the new structure on the new HD.
I have an Apple TV that was syncing to a Mac Mini that had a defective HD and logic board. I was unable to backup the Apple TV to the Mac Mini for some time before I had the mini repaired by AppleCare.
When the mini came back, it registered on the Apple TV as a new computer and I couldn't sync with it without losing hundreds of dollars of content.
All was okay until the other night when Apple's defective update for the ATV erased ALL my content. I updated it to 3.1 this morning, which Apple indicated might fix the problem. It didn't. The old content is all gone.
My ATV is out of warranty, so I removed the old HD and copied the files from the "Purchased" folder to my mini. That saved most 101 items that I purchased since 12/08 (but not all for some reason). I lost all content prior to 12/08 that I purchased, as well as some content since 12/08 that I purchased.
I need to keep the old HD untouched until I can purchase data recovery software. I'd like to just put a spare IDE HD in that I have into the ATV and just start over with what is in my Mac mini iTunes.
So, how does one do that? I'm not gifted with vast Unix knowledge or experience and I don't want to clone the old HD and risk losing more data if it writes to the HD for some reason.
Thanks.Hi Atagahi,
Firstly, have you emailed Apple iTunes support indicating your issue? There have been instances in which Apple will allow re-downloading of all content you've purchased on the iTunes store.
Secondly, I highly recommend you get an external usb hard drive, attach it to your mini and use it as a Time Machine drive to keep your iTunes purchases safe. It's a fact of life that all hard disk drives will eventually fail. The Apple TV should never be used as any kind of permanent storage space; think of it as a 'temporary holding' zone. And as you've unfortunately already experienced, a glitch wiped out all content on the Apple TV.
As regards your original question, it seems like there have been folks who've done just that...taken the internal drive in the Apple TV out and attached it to a mac and copied files from the Apple TV hard drive. I have not had to do such, so I cannot offer any help here except for Linda's suggestion on googling the net for solutions.
Good luck and hope you get all your purchased content back Atagahi!
Message was edited by: Alec -
I changed my Apple ID (new email address, same account), via the Apple website on my laptop, but when I try log into ITunes on my iPad, it still has the old ID. It is greyed out, so I can't change it. On researching further, it seems I have to sign out of iTunes and sign back in using new address. Ok so far, but when I do this, it asks if I want to delete the account and any data on the cloud. If I do this, and sign in with changed ID, will all the data be there Or will it be lost? Thanks for any advice.
Sounds like apps purchased from those accounts got synced to the iPod and it is asking to log into that account to update them.
- Apps are locked to the account that purchased them.
- To update apps you have to sign into the account that purchased the apps. If you have apps that need updating purchased from more than one account you have to update them one at a time until the remaining apps were purchased from one account.
What do yo see if yo go to the Updates Section of the App Store app? -
How to "Create New RSS Channel" in GroupSpace.
I try to create new rss channel following groupspace document( http://download.oracle.com/docs/cd/E13155_01/wlp/docs103/groupspace/index.html ).
but it have some error "Failed to add RSS channel. Be sure this URL returns valid RSS output.".
I put "http://rss.news.yahoo.com/rss/oddlyenough" in input box follow the document.
I missing some information except the document?
Thank you .
Ron.use the standard java system properties (either through command line or code)
http.proxyHost,http.proxyPort,
http.proxyUser,http.proxyPassword - If your proxy needs authentications
regards
deepak -
Problem in creating new communication channel template in IR
Hi experts,
I currently have problems with creating new communication channel template in Integration Repository. If I choose the New-Context (Adapter Objects-->Communication Channel Template) the dialog "CREATE OBJECT" is shown. After I have inserted the Name of my template I push the create button.
Then the following error occurs:
STOP: Internal error: Object cannot be changed in display mode
DETAILS: Internal error: Object cannot be changed in display mode(CANT_SET_MODIFIED).
So can anybody help me. Thanks for your help.
Best regards.
MarioHi Mario,
This is genral problem appearing in PI 7.0 with SP12.
Previously, i have the same problem.
i.e STOP: Internal error: Object cannot be changed in display mode
DETAILS: Internal error: Object cannot be changed in display mode(CANT_SET_MODIFIED)
Regarding this we raise an OSS note SAP people.
they gave the suggestion that update with SP13.
i discuss with basis team and install the SP13.
now the problem is solved, i can able to create the Communication Channel Template.
Please award points, if it really help to u
Regards
Mahesh. -
Problem access to Facebook news feed feature
Hi hi, does anyone encounter problem access to facebook news feed feature using iphone via WIFI? I noticed this error occured is after I installed the FB new apps just a couple of days. May I know is it due to system bug? Can anyone help? Many thanks!
That would be a question for Facebook support. If iPhoto is posting to your Facebook photos then it is up to your Facebook settings what to do next
LN -
HELP - I want a new feed that will pick up all my previous podcasts.. how??
Itunes has happily picked up all my Tory RAdio podcasts in the past. A couple of weeks ago it stopped - which is worrying.
Given that has happened I thought I could use the opportunity to create a new feed that will pick up all my previous podcasts (and obviously new ones as and when I do them) but also I can finally put some artwork so that Tory Radio on itunes actually has the nice logo I would like.
I previously just submitted www.toryradio.podbus.com into itunes which is where I host and it has picked them all up.
How for a complete technophobe - can I get something that I can submit to Itunes as a new feed that will pick up all the old podcasts, new ones and include a logo?
Is it easy? How can I do it?I can't check on iTunes until tomorrow morning, but in the meantime I can tell you what's wrong with your original feed. The top two lines read
<?xml version="1.0"?>
<rss version="2.0">
They should read
<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" version="2.0">
Without the itunes URL in the second line iTunes cannot parse the required tags to show your episodes. I would guess that the line was there originally but got removed, after which iTunes would not recognize new episodes. Put it back and reload the feed, and it should work if iTunes hasn't terminated your podcast in the meantime.
But to answer your original question: assuming you have access to both feeds; copy everything in the original feed from the first <item> to the last </item> near the bottom.
In the new feed, find the last </item> tag at the bottom (just above the </channel> and </rss> tags), start a new line just after it and insert the material you posted from the original feed (though there are an awful lot of episodes there, and though it's technically not a problem you might want to consider whether you really need all of them in the feed).
If you don't have direct access to the feeds but are using one of the podcast creation services which don't allow this, you will have to follow whatever method they provide for adding the episodes, possibly one at a time. -
Problem establishing SNA session over DLSw link
We are experiencing sporadic problems establishing an SNA session over a DLSw tunnel. The session is between a Tandem host and a CICS region on OS/390. A Cisco 7500 router performs one end of the DLSw link; that same router is channel-attached to the OS/390 mainframe using CIP and uses CSNA for SNA traffic.
The connection is initiated by the SNA software on the Tandem box, but it rarely works on the first attempt. The session goes into a pending state and has to be cancelled and re-tried. This has to be repeated until such time it is successful. Once the session is started it works without problem.
At the point of attempting to start the session all the components are in the 'correct' state, i.e.
-DLSw tunnel is connected
-Host PU is in 'connectable' (CONCT) state
-Host XCA defining CIP SNA and virtual lines are active
No action needs to be taken on the OS/390 end of the link (or anywhere else for that matter) between start attempts on Tandem.
From device traces on OS/390 my suspicion is that the session requests are not actually getting to VTAM on OS/390 for some reason. DLSw traces on 7500 appear the same in both case of success or fail of session start.
Has anyone experienced this, or know of problems in this area that may explain what we are seeing?
Thanks.
KeithMary,
thanks for the extra information.
I would advice at this point the following:
Open a case with the Tac. Provide the information you have at that point and agree on a action plan, i.e. what to look at what trace to take, when you are able to recreate the problem.
What i can see from the documents you attaches is the following:
I only see sna circuits in the dlsw circuit history.
In you show dlsw reach there are quite some netbios names aswell and a large number of mac addresses learned from the remote peer.
You may want to discuss some filter options with the tac to make sure we cut down on the size of the reach cache and to make sure we advertise only those mac addresses which are really needed.
From the show version, 12.1(17) should be fine for dlsw in the way you described it.
From the show dlsw circuit history detail we have a couple of times that circuits are disconnecting.
Most of the time these circuits are up for a long time and they get the following sequence before terminating:
Index local addr(lsap) remote addr(dsap) remote peer
1157627904 4000.0410.0001(08) 0800.8e00.9708(08) 10.19.2.124
Created at : 12:02:55.115 GMT Mon Nov 29 2004
Connected at : 12:02:55.447 GMT Mon Nov 29 2004
Destroyed at : 22:04:01.158 GMT Wed Dec 1 2004
Local Corr : 1157627904 Remote Corr: 3321888773
Bytes: 140092/145133 Info-frames: 1748/2327
XID-frames: 1/2 UInfo-frames: 0/0
Flags: Remote created, Local connected
Last events:
Current State Event Add. Info Next State
CONNECTED DLC DataInd 0x0 CONNECTED
CONNECTED WAN infoframe 0x0 CONNECTED
CONNECTED DLC DataInd 0x0 CONNECTED
CONNECTED WAN infoframe 0x0 CONNECTED
CONNECTED WAN infoframe 0x0 CONNECTED
CONNECTED DLC DataInd 0x0 CONNECTED
CONNECTED WAN infoframe 0x0 CONNECTED
CONNECTED DLC DataInd 0x0 CONNECTED
CONNECTED WAN infoframe 0x0 CONNECTED
CONNECTED DLC DataInd 0x0 CONNECTED
CONNECTED WAN infoframe 0x0 CONNECTED
CONNECTED WAN infoframe 0x0 CONNECTED
CONNECTED DLC DataInd 0x0 CONNECTED
CONNECTED ADM WanFailure 0x0 HALT_NOACK_PEND
HALT_NOACK_PEND DLC DiscCnf 0x0 CLOSE_PEND
CLOSE_PEND DLC CloseStnCnf 0x0 DISCONNECTED
this circuit was up for more than a month and the reason for disconnection was ADM WAN failure which means the dlsw peer went away at some point.
Some other circuits get these sequence:
CONNECTED WAN infoframe 0x0 CONNECTED
CONNECTED WAN halt-dl 0x0 HALT_PENDING
WAN halt-dl means the other end tells us to disconnect. No information on this end why. Most of the time it is a legitimate disconnect. but you would need to look at the dlsw peer to get information who terminated the session.
I dont see anything specific in a sense that a circuit did not go to connect at all. In the information you supplied.
Again my advice open a case with the tac and then you can work the issue to completion.
thanks...
Matthias -
I'm currently using Google reader for my feeds but Google is discontinuing it in July. So I need a new feed reader option. I am trying to set up the Brief add-on as my reader, using Live Bookmarks but when I attempt to subscribe to a feed, the subscribe button doesn't work.
Many site issues can be caused by corrupt cookies or cache. In order to try to fix these problems, the first step is to clear both cookies and the cache.
Note: ''This will temporarily log you out of all sites you're logged in to.''
To clear cache and cookies do the following:
#Go to Firefox > History > Clear recent history or (if no Firefox button is shown) go to Tools > Clear recent history.
#Under "Time range to clear", select "Everything".
#Now, click the arrow next to Details to toggle the Details list active.
#From the details list, check ''Cache'' and ''Cookies'' and uncheck everything else.
#Now click the ''Clear now'' button.
Further information can be found in the [[Clear your cache, history and other personal information in Firefox]] article.
Did this fix your problems? Please report back to us!
'''Try Firefox Safe Mode''' to see if the problem goes away. Safe Mode is a troubleshooting mode, which disables most add-ons.
''(If you're not using it, switch to the Default theme.)''
* You can open Firefox 4.0+ in Safe Mode by holding the '''Shift''' key when you open the Firefox desktop or Start menu shortcut.
* Or open the Help menu and click on the '''Restart with Add-ons Disabled...''' menu item while Firefox is running.
''Once you get the pop-up, just select "'Start in Safe Mode"''
'''''If the issue is not present in Firefox Safe Mode''''', your problem is probably caused by an extension, and you need to figure out which one. Please follow the [[Troubleshooting extensions and themes]] article for that.
''To exit the Firefox Safe Mode, just close Firefox and wait a few seconds before opening Firefox for normal use again.''
''When you figure out what's causing your issues, please let us know. It might help other users who have the same problem.'' -
I experience network error on my news feed in facebook accounts i try other facebooks account but still working only my facebook account experiencing this kind of problem what happen?
I have previously had this problem and it was a struggle to
recover. It has just returned for no apparent reason. I tried to
follow adobe and other methods to no avail. I then decided to close
ALL programs and applications as recommended by Adobe. This
included Windows Messenger, my security program Bullguard, Norton,
and even my printer link.
So my advice is follow Adobe's recommended settings for
Active X in internet options, close all running progs and the
installation should be OK. A restart to bring back normal
applications should see the end of the problem. -
Facebook News Feed not connecting on z10 FB app
Hi Guys - Help required (again)!!!
I've noticed a number of posts on here about the Facebook App not loading News Feed properly. I had this issue for about 2 days 3 weeks ago, and I have it again now
Does anyone have any idea what is happening?
The solution I was given last time (well reason, it didn't solve it) was that there was a problem with the API servers, which figures as the error on screen suggests a connection failure, but as every other FB screen is available to you (i.e. your own profile page, other FB users profile pages, settings etc.), it just won't connect to News, is it more specific than that? Live feed?
Any information appreciated
Happy Thursday
JonHey Jon,
Welcome to the BlackBerry Support Community Forums.
Thanks for the question.
Do you get any specific errors when loading the news feed? Can you also provide a screenshot of what is happening on the newsfeed if possible?
To take a screenshot press both volume buttons at the same time.
I look forward to your reply.
Cheers.
-ViciousFerret
Come follow your BlackBerry Technical Team on Twitter! @BlackBerryHelp
Be sure to click Like! for those who have helped you.
Click Accept as Solution for posts that have solved your issue(s)! -
Problem establishing a https connection
Hi,
I have the following problem in establishing a connection to a secured webserver.
I tried to establish a connection to a secured web server from a application running in weblogic. I used the weblogic specific APIs, created a weblogic.net.http.HttpsURLConnection. I set the connection properties like setSocketFactory and used NulledTrustManager and NulledHostNameVerifier.
I then tried to get a DataOutputStream so tht i can write my xml string to tht connection, nothing really is happening. When i tried to print the headervariables it is not printing any..... I am doubtful if the connection is established....
i will attach my code....plz tell me if i am missing something out....
The same code works when i use Java APIs....they are included in the code as comments....
Actually i have a jsp file in the secure server which takes this string and sends mail to the required recipients.....
Plz help.....Thanks in Advance for all the helping minds...
In the code, the trustallcerts and hostnameverifier are classes written similar to nulledtrustmanager and nulledhostnameverifier.... ie it certifies everything and trusts everything.
/* Code Snippet */
import java.io.*;
import java.util.Properties;
import java.net.*;
import weblogic.security.SSL.*;
import weblogic.net.http.HttpsURLConnection;
public class EmailClientW
String XMLMessage;
byte[] XMLMessageBytes;
String strURL= "some https url";
public EmailClientW(){}
public EmailClientW( String str )
XMLMessage = str;
XMLMessageBytes = XMLMessage.getBytes();
public void HttpsConnect()
try{
// Create the socket connection and open it to the secure remote web server
// Used in case of Java API
//System.setProperty("javax.net.ssl.trustStore", "C://bea/jdk141_03//jre//lib//security//cacerts");
weblogic.security.SSL.SSLContext sc = weblogic.security.SSL.SSLContext.getInstance("SSL");
//Used in case of Java API
//sc.init(null, trustAllCerts, new java.security.SecureRandom());
TrustManagerJSSE tManager = new TrustAllcerts();
sc.setTrustManagerJSSE(tManager);
//sc.loadLocalIdentity(certs, privateKey);
sc.setHostnameVerifierJSSE(new HostNameVerifier());
//Used in case of Java API
//javax.net.ssl.HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
URL url = new URL(strURL);
weblogic.net.http.HttpsURLConnection connection = new weblogic.net.http.HttpsURLConnection(url);
connection.setSSLSocketFactory(sc.getSocketFactoryJSSE());
connection.setHostnameVerifierJSSE(new HostNameVerifier());
System.out.println("CLASS " + connection.getClass().getName());
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestProperty("Connection", "Keep-Alive");
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type","text/html");
connection.setInstanceFollowRedirects(false);
connection.setAllowUserInteraction(false);
connection.setHostnameVerifierJSSE(new HostNameVerifier());
//This doesnt print anything.
System.out.println("getOutputStream :"+connection.getHeaderFields());
DataOutputStream dos = new DataOutputStream(connection.getOutputStream());
dos.write(XMLMessageBytes, 0, XMLMessageBytes.length);
dos.flush();
dos.close();
}catch (java.net.MalformedURLException mue){
System.err.println("\n" + "URL is not exist or protocol does not exist or there is a typo in the submitted URL" + "\n");
}catch (java.net.UnknownHostException uhe){
System.err.println("\n" + "Remote server does not exist in DNS." + "\n");
}catch (java.io.IOException ioe){
System.err.println("\n" + "I/O Exception in the connection try again or contact developer." + "\n");
ioe.printStackTrace();
}catch (Exception exp){
System.err.println("\n" + "Exception in the connection try again or contact developer." + "\n");
exp.printStackTrace();
public static void main(String args[]) throws Exception
new EmailClientW().HttpsConnect();
}Thx for the prompt reply....But this is not solving my problem.
I am trying to establish the connection from inside a weblogic container. So i am forced to use weblogic APIs.
Also i use NulledTrustManager and NulledHostNameVerifier so i think i neednot worry about the certificates...
Weblogic APIs donot contain init method for sslcontext....So if u can see the code, i have commented these lines and set these properties using the Weblogic supported APIs.
Also the method which u specified (Using Java APIs) was working for me....problem arose when i tried to use weblogic specific APIs.
Can u go through the code once and tell me if i am missing something over there? Coz header is not getting populated, the request is not hitting the secure server
One more thing, if i use the HostNameVerifier(dummy one to verify all the hosts), it tells me tht the key chain obtained from the host is incomplete.... and throws javax.net.ssl.SSLKeyException...
If i remover hostnameverifier, the exception is not thrown, but the request is not hitting the server...
Plz help me ... i am helplessly stuck in this situation...
Thx in advance for all the helping minds.....
Maybe you are looking for
-
How to include field in document header
Dear All, Any one please give me some input on how to include a field in document header of f-02 . Thanks, Srini.
-
Can anyone recommend where to sell my iPad 2 in the UK apart from EBay?
Hi folks, I have an iPad 2 32GB WIFI in black, immaculate condition with Smart Cover and Camera Connection Kit. Just wondered if anyone could recommend anywhere apart from eBay? Many thanks, Chris.
-
I have a iPad, 32g, and for no apparent reason, the screen goes black and whatever app I am using is closed on me. It seems to be happening more and more frequently, especially when loading apps, or trying to watch a YouTube video. I try to open saf
-
[SOLVED][AUR] lib32-allegro Missing dependencies...
Hello community... still on my first week on ArchLinux so bear with me... i find this strange... im trying to install a game Beat Hazard Ultra (i have the bin file) and beathazardultra can be found in the aur (game separated) i looked at the PKBUILD
-
Disappointed with the iTunes store error.
I always loved having apple products, but after this newest error i'm extremley disappointed, I shouldn't have to keep trying to get in the app store when it basically keeps saying 'cannot connect to iTunes store' I have around 4 apps that need to be