Life after the panic protocol

We have 2 servers that run on GridGain containing Coherence distributed caches (version 3.4.1). The nodes in both servers are used for processing events. Those events are stored in the cache.
When the network connection between server A and B fails, each one will continue in its own cluster island. Once the connection is established, Coherence will first log a message like the following:
10 Mar 2009 10:07:32,748 [Logger@9257178 3.4.1/407] WARN Coherence - 2009-03-10 10:07:32.747/88550.631 Oracle Coherence GE 3.4.1/407 <Warning> (thread=Cluster, member=1): The member formerly known as Member(Id=6, Timestamp=2009-03-10 09:07:40.389, Address=192.168.1.7:8088, MachineId=40455, Location=process:29423, Role=ServerMain) has been forcefully evicted from the cluster, but continues to emit a cluster heartbeat; henceforth, the member will be shunned and its messages will be ignored.
and half a minute later it will log the following:
10 Mar 2009 10:08:02,803 [Logger@9257178 3.4.1/407] WARN Coherence - 2009-03-10 10:08:02.803/88580.687 Oracle Coherence GE 3.4.1/407 <Warning> (thread=Cluster, member=1): An existence of a cluster island with senior Member(Id=6, Timestamp=2009-03-10 09:02:07.28, Address=192.168.1.7:8088, MachineId=40455, Location=process:29423, Role=ServerMain) containing 5 nodes have been detected. Since this Member(Id=1, Timestamp=2009-03-09 09:31:52.149, Address=192.168.1.6:8088, MachineId=40454, Location=process:6853, Role=ServerMain) is the senior of an older cluster island, the panic protocol is being activated to stop the other island's senior and all junior nodes that belong to it.
All this makes sense. However there's about 30 seconds between the time the network connection was reestablished and the time the cache from the "bad" cluster island was restarted. During those 30 seconds we are already assuming that the nodes from the "bad" cluster island can be used for processing, so events are already added to the cache on the nodes of the "bad" cluster. After the panic protocol the caches are restarted and the events that were added in those last 30 seconds are gone.
There are two solutions that come to my mind.
1. We make sure that we don't consider those rejoined nodes for processing events untill after the panic protocol is resolved. Could we use a MemberListener for that? Will we only get a MemberListener.memberJoined() after the panic protocol is executed?
2. We already use those rejoined nodes for event processing, but we restart any event processing once we get notified of the occurence of the panic protocol. Is there a way we can listen for such an event indicating the cluster has been restarted?
Best regards
Jan

Hi, the problem is we don't know in advance how many members the cluster will contain or even how many nodes each server will contain or how many servers there will be in the cluster. So stopping event processing when the amount of members in the cluster drop to a certain level won't work.
However we could keep a list of the servers that aren't available anymore and when the connection is reestablished, wait for the members to reappear in the cluster before considering them for event processing.

Similar Messages

  • I Have been getting BAD battery life after the ICS UPDATE !! on the DROID RAZAR MAXX have this happen to any one else ??????

    i have been getting really bad battery life after install ICS on my droid razar maxx i can only go 1 day and 3 hours where i used to be able to go 2 day easy have this happen to anyone else can someone tell me how i can make my battery life better??????/ and do i need to let my phone go complete dead before fully chargeing it again ??????
    before install the ICS update i had 30 percent of battery life remaining so i pluged it in and let charged for a little bit while it updated my phone and ever thing after it updated i unplug my phone and started playing on my phone to see what ICS was all about but when it got to 15 and said " connect to your changer so i did !!!!! should i haved let my phone going complete dead since i unplug without it fully changer or does it matter ?????????????? if someone could please help me !!!!:)

    What was your battery reading right after the update when you unplugged the phone? I have never let my Maxx drain further than 20% battery ever before I charge it. Yesterday after the update I also played more with the phone trying to learn the OS, as did most everyone on this forum. Give it a few days of your normal use before you decide your battery life has decreased. I took my phone off charger today at noon, it was 100%, and now it's 80%, with about 30 min talk time, and mostly standby. that's about normal batt life for me, maybe a little better than before the update. I'm still going to watch it for a few days before I make a conclusion about battery life.

  • Life after the Treo 680 - Ive had enough of Palm

    Ive had enough of Palm Im afraid.
    I started life with a Palm V, then Vx then Zire.
    Because my Palm Desktop has so much critical info on my customers I bought a Treo 680.
    Now its keyboard doesnt work, I cant even turn the bugger on - I have to reinsert the battery to get the screen to light. 
    The touch screen still works but thats useless cause you cant do anything - like make a phone call.
    So Ive had to put the Sim Card into an old I-mate Jasjam - hate that phone and of course no contact info at my finger tips.
    I bought the Treo 680 cause it sync'd all my Palm Desktop but after only 6 months its stuffed.
    Where should I go now?
    Can I export all my Palm Desktop contacts to Outlook?  I cringe at the thought of using Windows Mobile OS.
    Open to suggestions ...... thanks from down under.
    Post relates to: Palm Z22
    This question was solved.
    View Solution.

    Install the free NVBackup to your 680.  This program is a real life-saver and has saved my butt many times!
    http://www.freewarepalm.com/utilities/nvbackup.shtml
    It will let you make an exact duplicate of your device on the card, can do it "automagically" if you desire!   I install and evaluate many new programs each week, and I have had to restore with NVBackup when some new program suddenly scrambles years of hard work and stability in my Palm Devices!     When you install the program, it will also install a copy of itself to your SD card.
    WyreNut
    Post relates to: Centro (AT&T)
    Message Edited by WyreNut on 02-19-2009 04:31 PM
    I am a Volunteer here, not employed by HP.
    You too can become an HP Expert! Details HERE!
    If my post has helped you, click the Kudos Thumbs up!
    If it solved your issue, Click the "Accept as Solution" button so others can benefit from the question you asked!

  • Life after the C7 Anna update

    Ok so where do I start……oh yes the Anna update
    So I updated my C7 to Anna (including the 1/2 and 2/2 subsequent updates) and this made my phone much sleeker and that is always a good thing in my book, NFC and some fancy new extras are also much appreciated….thumbs up for Nokia…. 
    However, since the update many things no longer work and some things have disappeared completely:
    1)      The phone now restart every few hours, usually when I’m not doing anything with it (phew), so while its not a direct inconvenience, no doubt it is draining my battery (not to mention all my home widgets need to be restarted/loaded each time)
    2)      Many of my pictures/images no longer appear in my ‘photos’ regardless of what folder they are stored (some appear and some do not and they reside in the same folder!!)
    3)      The music player editing is still not available grrrrrr (maybe I hoped to much for that one)
    4)      When I try to purchase something on OVI it gets to the confirmation of payment and then gives me an error message saying something like ‘the purchasing system is currently not working’ ?????? wow your losing money on that one ?????
    5)      Certain screensavers have vanished like the music player (song playing) and the slide show, why why why, they were great?
    Ok a little wrapping up is needed; I have always chosen Nokia in the past mainly because I believe Nokia phones are practical and as such can do all the things that I need from a smart phone.  While one alternative is an iPhone, I believe it to be extortionately priced with some seriously basic flaws (flash player missing, Bluetooth limitations, tied to iTunes, cannot remove MC or battery etc).  However, given the amount of problems I’ve been having recently I am somewhat tempted to start looking at android phones for my next purchase.
    I say this not as a threat or out of anger but out of a ‘what happened Nokia’ kind of disappointment.  Anna/Bella may not be enough if you can’t get your software in order, especially the variety of apps/games available on your market place, which is a pity really because your new phones are more than capable of such software! (for example, why are there no music sequencers/creators available?)
    Can anyone help with any of these problems? Or is anyone having other problems that I have not included here? Will Nokia even read and act upon this post? Tune in next week to find out……..
    P.S. spoke to support and their answer was to reinstall Anna and this didn’t work, alternatively wait for a new firmware update, and when that will be I don’t know.
    Solved!
    Go to Solution.

    Hi all, this is just an update on some of the things I have resolved from my original post 1) Unfortunately the restarts are still happening, usually when the phone is idle for a period of time (around 2-3 hours) yet it doesn't crash or reset when in use. From what I can gather some people believe it to be hardware (sent it back) and some think it is software (wait for update). but this restarting never happened like this before the Anna Update. phonehacker has recently posted this which i'll try tonight Thanks /t5/Pool-of-Knowledge/My-C7-restarts-very-often-Wh​at-should-I-do/td-p/1014817 2) I have resolved the missing images from 'photos' issue - After Anna update I moved all my pictures from the Pictures (on memory card) folder to the Images folder (the one where camera pics are stored, still on the memory card) and now they all appear in the photos app. 3) No news on the music update (allowing editing) but I hear this is an issue with all S^3 devices, so patients is needed for this one. 4) I can still purchase on the OVI store as long as I am connected via WiFi, but not through 3G, any ideas? 5) I hear it is a given that the extra screensavers have been removed Hope some of this helps, or that someone can resolve my restarting issues. Thanks all ***edited this three times and for some reason spaces are not appearing, sorry for the way it reads

  • Life after the DiveLog tutorial

    I have been learning java for a while now, I have studied sun's Java Tutorial and other similar sites. Then after getting a grasp of the basics I moved on to example applications, such as the DiveLog application. My question is 'where do I go from here'? I was wondering if any readers know of any online example application tutorials (similar to the DiveLog or maybe slight more advanced). I found this method of learning more beneficial, (showning the principles being put in action). I am particularly interested in seeing how a java project develops from design to implementation using as much theory as needed and putting it in a form that is interesting to read and practice. Any links to similar projects would be greatly appreciated.
    Incidentally, I am aware the DiveLog tutorial is not yet complete (only up as far as part 4). So if the authors of the tutorial happen to read this article, it would be worth letting the readers know when the next parts will be posted.
    The divelog tutorial has been an excellent tutorial to learn from. Its method of explaining concepts as they were being implemented in a real project provided a more interactive and interesting way of learning than simply explaining theory page by page like most. Hope to see the next parts soon! If there are other similar tutorials out there, in terms of the method of teaching, on whatever subjects of java, please let me know. Thanx for any help.

    This may not be exactly what you are looking for, buy why not consider volunteering some time to help write java for a project that interests you?
    I learned alot about java by volunteering to help write java code for an internet game. It was a fun way to learn, the people were very helpful with any questions I had, they also gave me space on their server, gave me software, etc....
    If that sounds like something you want to look into, check out http://sourceforge.net/
    Look over the list of current projects, maybe something will interest you.

  • The panic protocol and Coherence 3.5

    All,
    We just upgraded from 3.3.1 to 3.5 but I'm having trouble forming a cluster in multi-server environments. Our config files were developed against older versions of Coherence and I had a lot of trouble with them at first, some of which is detailed here: Config file problem with new Coherence 3.5
    The problem now is that we have 2 standalone nodes and 2 application nodes (WebLogic) spread across 2 physical servers (1 standalone and 1 application on each box.) Previously (Coherence 3.3.1,) they all formed one happy cluster of 4 members. Now (Coherence 3.5,) they form separate clusters: each physical machine makes a cluster of 2 members. At startup, I can see the 2-node clusters form. Some time later (not immediately) I see the "unexpected cluster heartbeat" message warning about getting a heartbeat from the other physical server. Clearly the members of the different servers can communicate to some degree if they get these unexpected heartbeats. But why don't they form a cluster in the first place?
    If I understand the config correctly, we're using a ttl of 4, the default. I ran the multicast test and a ttl of 1 worked also. I think the join timeout is 30000.
    When the standalone node starts, it outputs a ttl of 4 and the expected cluster address and port to the log.
    One wrinkle in the config is that there are 2 applications deployed to the same weblogic jvm that both use Coherence. They are in separate classloaders and use unique cluster ports. This hasn't been a problem in the past. Now, however, my app is Coherence 3.5 and the other one is still 3.3.1. The Coherence jars are not shared and the startup params apply to both applications.
    In the past I've seen errors where 2 nodes weren't using the same coherence version, same cluster name, etc. but I don't see anything like that now.
    thanks
    john

    Hi John,
    The clustering technologies did not change between 3.3 and 3.5. The fact that you could establish a multicast best cluster in 3.3 and not in 3.5 is therefor quite odd. My initial guess would be that your network may be blocking certain multicast address/port ranges? Are you using the same multicast address and port as you'd successfully used in 3.3? Also please use this address and port when running the multicast test to make it as close as possible to the medium on which coherence is trying to operate.
    If none of these suggestions resolves the issue, can you please post the following:
    - multicast test output from all nodes running the test concurrently
    - coherence logs from all nodes, including startup, and panic
    - coherence operational configuration
    Regarding the mix of Coherence 3.3 and 3.5 in the same JVM. So long as they are classloader isolated and running on a different multicast address/port you should be fine. Note I'm suggesting that both the address and the port be different. Some OSs (Linux) has issues related to not taking the port into consideration during multicast packet delivery. It wouldn't hurt to try starting 3.5 without the 3.3 app running, just to ensure that it isn't causing your troubles in some unforeseen way.
    thanks,
    Mark
    Oracle Coherence

  • I have a mid 2014 13' retina display MacBook Pro that went into kernel panic twice, the trackpad froze, and after the second kernel panic it will not turn on. This is a two month old laptop with no history of problems. Any ideas?

    I purchased it in December and three days ago it kernel panicked twice. After the second event, it will not boot at all. The charge cord is green when it's plugged in and seems to indicate the battery is fine. When it went into kernel panic, the trackpad locked up. I had a technician open it up to see if there were any obvious issues and everything looked ok visually. I can't boot it into safe mode or anything involving powering the machine on, as it is not responding to anything. I have a major test coming up in five days that I am taking on this laptop and am desperate for answers. Any information you can offer at all would be helpful.

    Try SMC and NVRAM resets:
    http://support.apple.com/en-us/HT201295
    http://support.apple.com/en-us/HT204063
    Then try a safe boot again:
    http://support.apple.com/en-us/HT201262
    Ciao.

  • Screen goes black, shut the lid for a few minutes, open the lid the turn on chime comes on and the screen comes back to life. The computer itself has continued to update just no screen. After a few minutes goes black again. Is logic board going bad?

    Screen goes black, shut the lid for a few minutes, open the lid the turn on chime comes on and the screen comes back to life. The computer itself has continued to update just no screen. After a few minutes goes black again. Is logic board going bad or graphic card issue? Trying to decide if computer is worth repairing as it was made in 2007.

    It does sound like you might have a faulty video connection. See if you can make a Genius Bar appointment at your local Apple Store.

  • HT201365 Hello . I have an iphone for which is currently IOS 7 updated. I have had some trouble with its batery life after IOS 7 update. I advised from the phone shop that I need to give it a factory restart and I made it. Now i have icloud activation loc

    Hello . I have an iphone for which is currently IOS 7 updated. I have had some trouble with its batery life after IOS 7 update. I advised from the phone shop that I need to give it a factory restart and I made it. Now i have icloud activation lock and I have no idea when I create an icloud account. I donr remember any related email or password. I am completely losttt
    Any help guys ? ( I am currently stuck with Nokia 2210

    Welcome to the Apple community.
    Unfortunately, unless you know the Apple ID, there is absolutely nothing that can be done, you cannot use your mobile device. You may be able to find your Apple ID at Look up your old and forgotten Apple ID

  • I had an iMac with Snow Leopard 10.6.8, I  downloaded and installed Lion, did not make me any questions. After the reboot I get this fatal error: Panic(cpu 0 ......) Kernel trap ......

    I had an iMac with Snow Leopard 10.6.8, I  downloaded and installed Lion, did not make me any questions. After the reboot I get this fatal error: Panic(cpu 0 ......) Kernel trap ......

    Please post the Kernel Panic report. You will find it at /Library/Logs/DiagnosticReports  per
    http://support.apple.com/kb/ht2546
    When we look at the KP report we may find some clues why it is occuring.

  • After the latest update when I try to turn on my computer I get this error: panic(cpu 1 caller 0xffffff80039bd555)

    Here is a picture I took of the screen. It happened right after the latest update. Anyone have any idea why or what I can do?

    Try a Safe Boot OS X: What is Safe Boot, Safe Mode? - Apple Support be patient as it may take awhile. If you can get started in Safe Mode copy and paste the panic report in this thread Mac OS X: How to log a kernel panic - Apple Support

  • What is going on the my battery life, after installing Mavericks, what a drain!!!

    What is going on the my battery life, after installing Mavericks, what a drain!!!

    You can try resetting the SMC and NVRAM.
    http://support.apple.com/kb/ht1379
    http://support.apple.com/kb/ht3964
    You can also try running the short and/or extended Apple hardware test by holding D button while booting up and then following the instructions.
    http://support.apple.com/kb/ht1509
    If that doesn't work (seems not extremely likely that it will) you can backup and try reinstalling a fresh OS.
    If that still doesn't work maybe the battery is bad.  If you don't feel like messing with all that just take it to Apple.  They should have a test they can perform.

  • What is the standard battery life after charge

    What is the standard battery life after charge?

    About 1,000 full charge cycles. Mac notebooks- Determining battery cycle count - Apple Support.

  • Hey, i dropped my macbook yesterdey, and after the first restart it s not funtioning anymore. it s showing me the kernel error. panic cpu .. i have no idea what causes this or what to do .. could you please help me . thx

    hey , i dropped my macbook yesterday, and after the first restart it

    hey , i dropped my macbook yesterday, and after the first restart it

  • I have two rotating arrows at the top of my screen next to the 3G symbol. Does anyone know how to turn this off b/c it is using all of my battery life? This started after the iOS 5 update.

    I have two rotating arrows at the top of my screen next to the 3G symbol. This happened after the iOS 5 update. Does anyone know what it is and how I get it off, it is draining my battery?

    At first read, I also thought it sounded a bit snide.  Who here has read the entire manual before going to the web for a quick search? (Not me, and in fact, I found my answer - to this very question, by reading this post WAY faster than if I had read the manual.  What is so wrong with that?) 
    That is the beauty of having support forums.  If it is a bother to answer, without being snide, one is not obligated to do so.  And my apologies if he was not being snide... but there maybe a better way to point out the manual, such as "I have included a link to the searchable manual which may be of assistance to you next time you are puzzled by your iPhone's behaviour". 
    Just my 2¢.

Maybe you are looking for

  • My mail app suddenly only allows one window at a time. Any ideas ?

    When using mail app I get to work on one window at a time and can not do anything with the app or open another window. this is only on by macbook. Any ideas

  • Palm Centro calendar changing synced items from Outlook 2007 exchange calender

    I use ActiveSync to sync my exchange server to my Centro.  Everything syncs fine except for recurring calendar items.  For example, I have a recurring meeting for the first weekday of each month.  Outlook shows it correctly -- but my Centro shows the

  • Dual 23 inch mouse disappears on Left monitor only

    Running Leopard 10.5.2 with Dual monitors. ATI Radeon 9800 card AGP on Dual G5 Mac 2.5 MHz I can see everything fine on both monitors and can drag windows to both monitors. The only problem is that the mouse disappears on the left monitor. But if I'm

  • Starting individual web-apps in an appserver instance

    I know this is against the J2EE spec but surely Sun did not expect 100 apps deployed in one JVM instance (even if it is load balanced) and expect all the apps to be restarted when I just want to restart one app. Is there a way to restart a single web

  • Clustering the SQL 2005..

    HI Expert!! I am in preparation phase to install the MSCS there are some pre-requisites which should be met first, Q1 You have created domain user groups for each clustered service like SQL Server, SQL Agent, Full Text. A1 Yes. I have created in my d