ISE MnT Node purge running for 6 days straight

Hi all,
Our ISE 1.1.4 patch 2 MnT node appears to be stuck in DB purge. I am getting e-mail alerts that say "Hourly purge skipped as purge is already running." Also, when I try to run a backup of the MnT node I receive the message. "Cannot submit full backup when data purging is in process."
We had received the "maximum open cursors exceeded" error. When that I happened I re-synchronized the deployment which re-started the services on the MnT node. This cleared the open cursors error but left us where we're at now. I was hoping it would clear itself with time, but it hasn't.
All I can think to do is restart the ISE services on the MnT node, but I'm a little worried about what might happen if I do that in the middle of a purge. Of course we don't have a recent backup of MnT (see above) and I would not like to lose the historical data.
I haven't opened a TAC yet, will if no answers here. We can't patch or upgrade above 1.1.4 patch 2 because we're waiting on a fix for an unrelated bug.
Any ideas appreciated, thank you.

Hi Leroy Plock,
Let me explain you the root cause of experiencing the "maximum open cursors exceeded" .
In ISE 1.1.4 Patch 1 we have introduced a new hourly purge Mechanism. Due to this feature we are experiencing this open cursors issue.
The issue 'ORA-01000: maximum open cursors exceeded' is caused because of the feature of HOURLY PURGE introduced in ISE 1.1.4 Patch 1.
On each hour a purge process is triggered and a connection to the MNT database is opened. As per the  logic this opened connection should be closed right immediately after the transaction of purge process is completed. But this opened connection is not being closed and thus in a day 24 connections are kept opened.
In oracle we have set the count of 1500 open cursors and so the database will not open the cursor count beyond this value.
With the above said 24 connections opened every day this 1500 cursors will be consumed within 62 days (1500/24) and this error will then populate.
If we restart the MNT node once in a month these cursors will get freed up and then will not see this open cursor issue. This defect is addressed in ISE 1.1.4 Patch 4.
The defect for this issue is CSCuh70984

Similar Messages

  • Server goes into constant Full GC loop after running for a day

    This is a servlet application running under Resin, with Java hotspot 1.5_13 on Linux 64bit OS. JVM args are:
    -verbosegc -Xms8G -Xmx8G -XX:NewSize=512m -XX:MaxNewSize=512m -XX:+TraceClassUnloading -XX:+UseParallelGC -XX:PermSize=512M -XX:MaxPermSize=512M -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+DisableExplicitGC
    After running for a day or so, the server goes in to constant Full GC every few seconds, essentially hanging the server. Here is an excerpt of the JVM log:
    127731.579: [Full GC [PSYoungGen: 156635K->0K(352576K)] [PSOldGen: 7751937K->2260734K(7864320K)] 7908573K->2260734K(8216896K) [PSPermGen: 62855K->62855K(524288K)], 5.2972428 secs]
    127737.596: [GC [PSYoungGen: 179328K->157147K(351040K)] 2440062K->2417881K(8215360K), 0.1275439 secs]
    127739.066: [GC [PSYoungGen: 336475K->155824K(355136K)] 2709605K->2683786K(8219456K), 0.1729460 secs]
    127740.502: [GC [PSYoungGen: 340464K->169143K(353792K)] 2980821K->2836693K(8218112K), 0.1914048 secs]
    127741.562: [GC [PSYoungGen: 353783K->166305K(345344K)] 3021333K->2860977K(8209664K), 0.1367291 secs]
    127742.973: [GC [PSYoungGen: 340908K->153430K(349568K)] 3147976K->3124907K(8213888K), 0.1970560 secs]
    127743.946: [GC [PSYoungGen: 328278K->156065K(349568K)] 3299755K->3277928K(8213888K), 0.1868634 secs]
    127745.498: [GC [PSYoungGen: 330183K->156854K(349568K)] 3564442K->3545105K(8213888K), 0.1836370 secs]
    From the log it seemed that the memory usage jumped by 150M to 250M within 1.5 second, even when the user traffic was very light (a couple of page views a second, vs. 10 page views a second at peak). The same app was running fine for many hours on a Windows dev machine in load testing condition with about 300M heapsize. No memory leakage was found so far. (it would be pretty unlikely to have such a large memory leakage without being noticed in dev and testing)
    Thanks for any help.
    Edited by: tuxzilla on Nov 16, 2008 11:43 PM
    Edited by: tuxzilla on Nov 16, 2008 11:46 PM

    A NewSize of 512m seems small for an 8G heap. This would probably cause most objects to be rapidly tenured into the OldGen heap space. (Ultimately depends on your application.) You might consider either using a NewRatio in place of NewSize/MaxNewSize or not use either and instead see how the jvm handles things itself with"-XX:+UseAdaptiveSizePolicy"
    VisualVM is a good graphical tool for watching gc activity.
    NewRatio=1 is the largest you can go for Eden, which will set (Eden and survivor spaces) equal to oldGen space. ie, for 8G, 4g each.
    Watch activity in the gc tool, you will see "sawtooth" activity. If your sawtooth pattern does not maintain a flat profile, then you have in fact have a memory leak.

  • Help!!!!!! my iPod touch nd gen will not turn on at all even after charging it for 2 days straight what's wrong???

    Help!!!!!! my iPod touch nd gen will not turn on at all even after charging it for 2 days straight what's wrong???

    Try:
    - iOS: Not responding or does not turn on
    - Also try DFU mode after try recovery mode
    How to put iPod touch / iPhone into DFU mode « Karthik's scribblings
    - If not successful and you can't fully turn the iOS device fully off, let the battery fully drain. After charging for an least an hour try the above again.
    - Try on another computer
    - If still not successful that usually indicates a hardware problem and an appointment at the Genius Bar of an Apple store is in order.
    Apple Retail Store - Genius Bar       

  • A job in Cisco Prime Infrastructure 2.1 has been running for 50 days

    I found a job (SAM_Poller_22_15_GETVPNPoller_1878262177) in cisco prime infrastructure 2.1 (Administration -> Job Dashboard -> System-defined), which has been running for about 50 days. Is this normal? How can I stop it? Thanks for any help..
    SAM_Poller_22_15_GETVPNPoller_1878262177
    PollerJob
    In-Progress
    Running
    2014-09-19 01:28
    49 Days 01 : 28 : 37

    Not normal, seems to be stuck. Try to stop (ncs stop) and start (ncs start) the system processes, which should stop this job.
    If it happens again, i would recommend to open a TAC case for further investigation.
    -Thanks
    Vinod

  • When  upgrading to R12.1.1  script apilnupg.sql is running for 4 days

    Script apilnupg.sql ran for almost 7 days and errored with unique constraint error,support given datafix and restarted again
    now it is running for almost 4 days ...whether any performance issue of server wheter any specific init parameters need to set
    and another R12.1.3 instance with 11.2.0.2 DB is running on same server
    Solaris 10 -- DB and CM/admin ---
    Linux --forms/web
    Can anyone please help me

    I applied latest patch and tried to run patch again but still script taking 4 days to complete
    Whether it is performance of server ,hardware ,already other instance R12.1.3 running on same server
    our solaris sparc III 900 MHz

  • DiskWarrior running for 5 days now

    Here's my story.  My Timecapsule 2 TB WD Caviar Green HD failed.  It had/has roughly 1.25 TB of data in it so I tried data recovery software but it kept having an unmounting error.  I took it in to get repaired and they replied:  "A Gillware technician has completed the evaluation phase of the recovery process and determined that your drive is mechanically healthy, but will require a Tier-2 logical recovery considering the drive's catalog file is severely corrupted and will require us to try to figure out where all of the binary data resides within the drive."
    The free quote they gave me for this evaluation was really expensive ($1200) and I'm a student so I got the HD back.  I found out that they had actually opened my HD but for reasons I don't understand, it didn't have an unmounting problem anymore!
    Fast-forward to now, I've been running diskwarrior for 5 days now and it's been chugging along slowly at step 5 (as seen below).  It hasn't stalled because the "Speed reduced by disk malfunction" number keeps increasing (up to 26k+)  which according to the disk warrior website is the sign that it hasn't stalled.  My question is:  how much longer should I anticipate DW running for?  Has anyone else gone through a similar situation? 
    Any advice would be much appreciated!  Thank you so much for your time!!!

    The corruption of a drives catalog file is a logical error, not a mechanical one. A logical error can be caused by improper removal from a computer or sudden power loss.
    One of the misconceptions that circulate on ASC is the idea that a journaled HFS volume can be corrupted by a forced unmount, as for example in a system crash or a power failure. That idea is false.
    With filesystem journaling, when the volume is remounted after a forced unmount, the directory is automatically repaired, and files are restored to their previous state, using redundant information (the journal) stored on the drive. This is very well established technology, and for all practical purposes, it always works, unless the journal itself is corrupt due to a hardware failure in the drive or elsewhere in the data path. So if the directory of a journaled volume is found to be corrupt, the storage device hosting the volume has thrown at least one unrecoverable hardware error.
    People who don't understand journaling will tell you to repair the corrupt volume with Disk Utility (if possible), or with Disk Warrior otherwise, and then go on using the drive as if everything is fine. That might have been good advice years ago, before the introduction of HFS journaling, when it was common for volumes to be corrupted as the result of forced unmounting. Today, because forced unmounts no longer cause that kind of corruption, it's relatively rare — or should be — and when it does happen, the implications are different.
    My opinion is that when a drive throws such a severe error that a volume directory not only can't be restored by journal replay, but also can't be repaired by Disk Utility, that drive should no longer be trusted with data. It should be taken to a recycling center.
    But even if you do choose to go on trusting the drive, if you have backups, you shouldn’t try to scavenge the directory with Disk Warrior. You should erase the volume and restore the data from your backups. Doing that is faster, safer, and cheaper than using DW. DW should be reserved for those situations where there are no backups, and even then, the drive should still be erased, or preferably retired, after the data is recovered.
    Deric Horn, the head of filesystem engineering at Apple, gave a presentation at WWDC 2012 in which he stated, with regard to journaled HFS (starting at offset 3:18 in the recording):  
    The journal means that you can pull power at any time, and your filesystem is still intact. There shouldn't be any data corruption or any volume corruption on that disk.
    You can hear the presentation by registering for a free developer account with Apple and going to this page. (Eventually the link will break; if that's already happened, you'll just have to take my word for it.) Look for "What's New In The Filesystem" near the bottom.
    If a poster on this site knows more about HFS than the Apple software engineer in charge of it, then the latter should be fired, and the former should get his job. Until that happens, I suggest you believe what Horn says.

  • Keeping imac on for numerous days straight.

    I use my elgato eye tv to record tv shows and then I convert it to ipod format so that I can watch it on my ipod video. However, converting numerous tv shows takes a while so I leave my imac on overnight quite often. My question is: Is it harmful to the imac if you keep it on running, for example, 3 days straight? I usually keep it on overnight but when I wake up I just jump right back on it and dont shut it down. And when I get back to bed I do the same thing. I check the temp of the cpu and it never really goes over 63.7 C. Just wondering so I know im not harming my computer in the long run because I've been doing it for quite a while now.

    Danny, according to the G5 20" iSight Manual, page 9:
    "When you have finished working with your iMac G5, you can put your computer to sleep or shut it down.
    If you will be away from your iMac G5 for less than a few days, put the computer to sleep. When your iMac G5 is in sleep, it's screen is dark & a small white light below the lower-right corner of the display pulsates. You can quickly wake your iMac G5 & bypass the startup process."
    On page 10:
    "If you will not be using your iMac G5 for more than a few days, shut it down."
    G5 20' iMac w/iSight/Remote, 2.5GB RAM, 250GB HD Mac OS X (10.4.4) OS 10.4.4

  • Problem in process chain running for 3 days

    Hi Experts,
    Please help me
    We have process chain like below
    1.     delete index
    2.     full loads going 3 cubes (16 infopacs)
    3.     loading to ods and then update to cube
    4.     delete overlapping requests (16 infopac)
    5.     create index
    6.     aggregate filling
    7.     delete PSA
    Now the issue is, from last Saturday onwards
    Process chains was still in Yellow status till date, so now, 1,2,3,4, are still running
    In process chains, out if 16 infopacs, 12 are green, remaining 4 are stopped
    And in ODS activation of ods data variant shows yellow, but when checked the data is activated, but not pushed to cube. When checked log it show u201Csystem exception error_messageu201D, so process terminated.
    When we checked for shortdump,
    It says u201Clocal time on the application server is incorrectu201D
    Experts please let me know how to solve this and correct the load.

    Hi Experts,
    Thank u for your help,
    I have done the steps manually yestarday, checked todays, still it is in yellow status.
    let me explain the scenario,
    The data is going to 3 cubes,
    the process chain is designed like this,
    1. initially delete index
    2. full load to cube using 16 infopac and loading from (2)ODS to cube.
    3. delete overlapping requests
    4. create index (3 cubes)
    5. fill aggregates (only one cube)
    6. Delete PSA request
    so now , chain is having problem at
    1. data is loaded to ods
    2. data activation (showing yellow, but when checked data is activated)
    3. further update to cube  (2 cubes) in process chain this is not started atall ( so y'day I've done manually)
    4. out 16 infocpack, only 12 infopacs are green and remaining 4 are not started.
    so yestarday I've manually schedule remaining 4 infopacs.
    and deleted previour requests in infocube as they are full load, but I did'nt done create index and fill aggregate steps.
    now the problem is , out of 3 cube report available request is present for 2 cubes, but for 3 cube  this report request is not available, but QM status is green
    please let me know how to proceed.
    one small silly question, do i need to  fill aggregate step to get this report available request.....please suggest.
    Thanks & Regards,
    Raghu

  • Unable to open database connection after applicaiton is running for several days.

    Has anyone experienced a similar error to this:
    After about 3 days of use our application starts to report errors opening
    the database connection. By that time we've had thousands of transactions
    happen. Oracle is only showing a few open connections to our iAS host so
    the error seems to indicate the connection pool. I'm configured for 64
    connections (the default). We are using the Oracle native driver on iAS
    SP3, Solaris.
    The iAS ksvradmin monitor gives errors when trying to see how many open
    connections it has (verified bug in SP3), so I can't get any info from iAS
    on the connection pool.
    Thanks in advance,
    Rodger Ball
    Sr. Engineer
    Business Wire

    iAS6 SP3 and earlier cannot detect dead connections. If connections become
    stale, iAS does not detect this and will hand out these stale connections.
    I don't know enough about your problem, but you can check for this.
    hope this helps,
    -James
    "Rodger Ball" <[email protected]> wrote in message
    news:9suucb$[email protected]..
    Has anyone experienced a similar error to this:
    After about 3 days of use our application starts to report errors opening
    the database connection. By that time we've had thousands of transactions
    happen. Oracle is only showing a few open connections to our iAS host so
    the error seems to indicate the connection pool. I'm configured for 64
    connections (the default). We are using the Oracle native driver on iAS
    SP3, Solaris.
    The iAS ksvradmin monitor gives errors when trying to see how many open
    connections it has (verified bug in SP3), so I can't get any info from iAS
    on the connection pool.
    Thanks in advance,
    Rodger Ball
    Sr. Engineer
    Business Wire

  • I too had the same vertical white line in the centre of the screen last night after having my iPhone 5 up and running for 10 days. I managed to turn it off by holding the buttons in and it came up with the iTunes screen....2 hours later it came good?

    I was talking outside on my iPhone 5 in a fairly humid environment (The Pilbara in January, 2014) then when the call was finished I hung up (as I was walking back inside to the air-conditioning) this white vertical line appeared in the centre of the screen and wouldn't go away. I couldn't use it NOR could I accept calls. (I called my number from my other phone). I eventually managed to turn it off after removing the protective casing (Otter Box) and then it came up with the white cord / iTunes screen........I left it for a while...it turned back on BUT is still had a flickering screen.......I re-charged it and I have backed it up this morning (at work) on iTunes and it seems to be working 'fine.' All I can say is 'DO NOT GET RID OF YOUR SAMSUNG GALAXY PHONE IF YOU HAVE ONE!' I haven't had an OUNCE of trouble with it and have owned it for at least 3 years. The iPhone has been up and 'running' since the 6th of January, 2014. This happened on the 15th of January, 2014.

    Plug your phone into the wall charger for at least 30 minutes...make sure you have a sim card in the phone...then:
    Leave the USB cable connected to your computer, but NOT your phone, iTunes running, press & hold the home button while connecting the USB cable to your dock connector, continue holding the home button until you see “Connect to iTunes” on the screen. You may now release the home button. iTunes should now display that it has detected your phone in recovery mode, if not quit and reopen iTunes. If you still don’t see the recovery message repeat these steps again. iTunes will give you the option to restore from a backup or set up as new.
    Make sure you have no anti-virus software running or any firewalls...turn all of that stuff off.

  • 1 I tried dowloading the free trial of indesign but it keeps stoping at 42 percent and won't budge. i have left it running for two days and there was still no progress. 2 I clicked on pin to notification in creative cloud so i could monitor the progress b

    need help

    If you had closed Creative Cloud application .
    Restart the computer once and check.
    If the Creative Cloud desktop app is launched ,
    Before starting installation of InDesign cc app.
    Clear temp folder.
    Please press Windows+R button together and you will get a run command window, please type-in %temp% and hit enter.
    Delete all contents present inside temp folder.
    You may skip the files which cannot be deleted.
    Then start the installation process for Indesign cc 2014.

  • 4G Inconsistent and down for 2 days straight (Downtown Boston)

    Hi there,
    I just recently purchased the iphone 5 and for the past few days I cannot get 4G network. My apps do not work on 3G hence why I paid the premium for the new 4G LTE enabled iphone 5. Is there a reason for 4G being down in Boston or is it just me?

    i have a 10gb HD and have roughly 4gb of free space left.
    Then that should not be the issue.
    my fan kicks in more often when i have the laptop plugged in. i was thinking it could be my power source, but i'm unsure.
    This is actually "normal" for later model TiBooks with two fans as one is dedicated to the power supply, but a 400MHz has only one fan, for the CPU, centered at rear. Still, I'm sure the heat from the power supply increases the temp of the whole unit, causing the CPU fan to run more.
    the HD won't be erased if i run that utility on it will it?
    No, not if you follow my directions. There are other buttons you can click in DU which will erase your HD, but they are clearly labeled as "Erase" and variations of that.
    I have a Norton's utilities CD i got when i got the laptop some years ago, but my Mac Tech at work told me it was somewhat useless for any OSX related problems.
    It's actually worse than useless; it could cause catastrophic issues with your HD. Whatever you do, DON'T USE IT. Some folks here would suggest going a step further and trashing that CD; if you know anyone who runs only OS9 or earlier (no OSX installed), you might give it to them, otherwise it really is useless if you don't use it and worse than useless if you do.

  • HT5262 hello i have a ipod 2g and it has been stuck on the connect to itunes for 3 days straight now what should i do

    I need help on my ipod touch 2g it has been frozen the last two days it froze on the connect to itunes logo

    Try a restart (I've had to do this myself, yesterday).
    If that doesn't work, try connecting it to iTunes (as it suggests).
    If that doesn't work, you could try waiting for the battery to drain and then the iPod will turn off. Then put it on charge as see if that sorts it out. (It should).

  • Missing records of all calls in detailed usage for 11 days straight, from 4 months ago!!

    I can see all detailed usage on both phones on my account, except there is absolutely no record of any calls, in or out on my phone from 10/29/12 - 11/10/12. It is very strange. My billing cycle ends on the 10th. I viewed and downloaded details from the billing period before and the billing period after, and can view and download spreadsheets of all of the calls for those periods. When I view or download the call details for the billing period 10/11/12 - 11/10/12 all the calls only go up to 10/28/12 no matter which way I view it, even on the bill copy. I never noticed before, but now I need the call records for legal reasons and that is why I noticed it now.
    Where did my call records go?

        Hello violets69
    I'm very sorry your missing call records. Lets get this addressed. Please send me a private message with your mobile number.
    JoeL_VZW
    Please follow us on twitter @vzwsupport

  • Upgrade to BW 7.4 runs for days in phase PARMVNT_SHD - Nametab Activation

    Hi everyone,
    We are upgradeing from BW 7.01 to BW 7.4 and facing a very long runtime in the phase PARMVNT_SHD (actual phase PARMVNT_TRANS). It has been running for 3 days!
    In this phase nametab activations are taking place. The upgrade is not stuck, but there are many objects to process from table DDXTT. I have checked the following:
    1. We are using the latest SUM tool SUM 1.0 sp10
    2. We are using the latest TP and R3trans
    3. Our database is on Oracle 11g, latest patches applied and parameters are set based on SAP note recommendations
    4. The hardware is HPUX with 8 cpus and plenty of memory.
    It is just very odd that this phase takes so long and that there are so many objects to process.
    Please help if you have any information or suggestions.
    Many thanks!
    Imran

    Hi Thorsten,
    We had exactly the same issue with our dev-system, it is very small.
    It was already running for more than 72 hours and it seemed the do nothing. The log files at ../SUM/abap/tmp/ where updated once in a while but not to much action.
    After setting the new parameters and restarting the DB it is running rally fast.
    Apart from yours we changed two more.
    This is the list of parameters we changed in our system (we will be setting these back as soon as the upgrade is completed)
    db_cache_size=2G (1G)
    shared_pool_size=1600M (800M)
    shared_pool_reserved_size=70M (40M)
    pga_aggregate_target=600M (400M)
    open_cursors=20000 (1000)
    I'll give you all an update when this is completed.
    Thanks to you all.

Maybe you are looking for

  • How do I get Photoshop Elememnts 7.0 to read my external hard drive?

    I have a signature 500 gp hard drive that pse 7.0 will not find or read.  In order for me back up photos I must do it through Digital Photo Professional or trhough Windows.  No matter what I try i cannot seem to get pse to read the external hard driv

  • Firefox for Android crashes often for me, I believe its because my tablet has a low amount of RAM.

    I have a low-end tablet that (despite having 8GB of storage), only has 512MB of memory/RAM. I've already encountered a few games that won't run because of this limitation. Up until now, it was things I could live without. But I get frequent Firefox c

  • Connection recv error in jolt connection

    Hi, We have Java GUI client connecting to the Tuxedo server thru Jolt. When we try to do some operation that connects to server after the GUI is idle for a about an hour we get Jolt error "Connection recv error" , sometimes we also get the error "Con

  • No discount when moving from Lightroom 5 to CC?

    I currently have Lightroom 5 and Photoshop CS5. Moving over to CC photoplan - is there no discount when you have perpetual licenses already?

  • Fields suddenly gone

    Post Author: hapax CA Forum: General I made a simple report that showed name, address, customer ID. That worked fine. Then I added some more database tables and their fields, removed the fields in the header, and added a text box with some text. Now