How do we prevent uncaught exceptions in our web role from causing 502's across our entire service?

We've recently migrated our web service to Azure. Part of it is running on an Azure Cloud Service. It serves around 5000 requests per minute, lots of dynamic image generation and various types of data feeds. We're currently running two Standard D4 instances
backed by a P1 SQL database and B4 Redis cache.
Our service is dependent on many 3rd party web services; try as we might sometimes bad data slips through the cracks and our service returns a 500 response (we output a short, custom xml response with usually an http 500 status). Occasionally this will happen
to routes that receive many requests - large networks or devices that aggressively retry - and this appears to cause Azure to return a 502 error page for all routes to our web role. It doesn't even really take that many. A few thousand over a 15 minute period
caused it today. It was simply an uncaught null reference exception.
This takes our entire service down and is a huge issue for us. Is an Azure load balancer in front of our services canning our incoming connections when it detects some level of 500 responses from our service? What should we be doing differently?

We're considering a couple of options. The first was simply to respond with a 503 response instead of a 500, and use the Retry-After header. We cannot guarantee clients will follow that header, however. Our other thought is to more closely monitor the error
rate (currently we have a job checking every 5 minutes for the total errors per end point over the last 15 minutes) and when some low threshold is passed return a 200 response with a short message or an "empty" response appropriate to the expected
content type (which might be a solid black image, for example).
If feels wrong to return a 200 response when our service cannot successfully complete the request, but we're not sure what else to do and what Azure may or may not be doing that is outside of our control here.

Similar Messages

  • How do I prevent nav links at bottom of site from wrapping?

    Mac OS 10.6.7
    Dreamweaver CS5.5
    How do I prevent nav links at bottom of site from wrapping? And how do I right justify the links?
    I changed something in CSS and now the footer links are wrapping. Previously, the links were at right/bottom of each page. When the browser window was made too narrow to display them, they would disappear along with the righthand side of the page. Now they wrap into multiple lines and expand out of the footer.
    HTML:
    <div class="footer">
          <span class="footerAlignLeft">&copy; 2011 Dorsay &amp; Easton LLP. All rights reserved.</span>
          <span class="footerAlignRt"><a href="index.html" title="Dorsay & Easton Home Page">  Home  </a><a href="index.html">|</a><a href="practice.html" title="Dorsay & Easton Practice Areas">  Practice Areas  </a>|<a href="team.html" title="Dorsay & Easton Attorneys">  The Team  </a><a href="contact.html">|  Contact Us  </a>|<a href="sitemap.html">  Sitemap  </a></span>
    </div>
    CSS:
    /* ~~ The footer ~~ */
    .footer {
         position: relative;/* this gives IE6 hasLayout to properly clear */
         clear: both; /* this clear property forces the .container to understand where the columns end and contain them */
         background-color: #CC6600;
         font-size: 10px;
         color: #FFC;
         padding-top: 10px;
         padding-right: 0;
         padding-bottom: 10px;
         padding-left: 10px;
    .footer a, a:link, a:visited, a:hover, a:active , a:focus{
         color: #FFC;
         text-decoration: none;
    .footerAlignRt {
         margin-right: 10px;
         font-size: 120%;
         position: fixed;
         left: 520px;
         text-wrap: suppress;
         text-align-last: right;
    Thanks!
    gotsowell

    Rather than posting code snippets, it's best if you upload your problem page and post a link here so we can view the live page in our browsers.

  • Have Operating System 10.6.8, Mail Program 4.6.  How can I prevent the next email in the que from automatically opening after I act on the previous email ? It creates big organizational problems for me. My computer changes this mode from self opening to m

    Have Operating System 10.6.8, Mail Program 4.6.
    How can I prevent the next email in the que from automatically opening after I act on the previous email ? It creates big organizational problems for me. My computer changes this mode from self opening to manually opening every few month with no ? action from me.
    Help

    Have Operating System 10.6.8, Mail Program 4.6.
    How can I prevent the next email in the que from automatically opening after I act on the previous email ? It creates big organizational problems for me. My computer changes this mode from self opening to manually opening every few month with no ? action from me.
    Help

  • In our web site the product sort order under our categories are random with Firefox but IE, Chrome is alright, any ideas?

    The sort order of products under our categories on our web site is random with Firefox, but the categories with products are in the correct order with IE, Chrome, etc. This incorrect order with Firefox occurs on various computers, i.e. Windows 7, XP, etc. and is not consistent. To make matters worse, sometimes it is random with Firefox and then returns to being correctly sorted but IE, Chrome, etc. are always in the correct order.

    Works fine here for me, even after refreshing.
    Start Firefox in <u>[[Safe Mode|Safe Mode]]</u> to check if one of the extensions (Firefox/Tools > Add-ons > Extensions) or if hardware acceleration is causing the problem (switch to the DEFAULT theme: Firefox/Tools > Add-ons > Appearance).
    *Do NOT click the Reset button on the Safe Mode start window.
    *https://support.mozilla.org/kb/Safe+Mode
    *https://support.mozilla.org/kb/Troubleshooting+extensions+and+themes

  • How can one prevent passing exceptions over an EJB-Webservice?

    Hello,
    I would appreciate if someone could give me some advice on the following. I tried to google it/use the search function, however, it was quite hard finding a precise search term.
    Let me try to explain the setting conceptionally without much code first.
    Three stateless session beans - let's call them BeanA, BeanB and BeanC - perform some specific tasks. Especially, they are using JPA to create and manipulate persistent entities. Clients like graphical user interfaces access these beans. I call them internal clients because they belong the same company like BeanA, BeanB and BeanC. Internal clients can know how to handle exceptions which are thrown in the beans.
    However, the functionality of these beans is also relevant to external clients, for example another company. A webservice (WebserviceBean) combines the functionality of the three beans and communicates with the external client. The latter one must not receive any exceptions thrown in my EJB application as this could leak implementation details but they must be informed, that the task could not be performed (some kind of "Please try again later"). As far as I understood (also tested it), JAX-WS takes exceptions and turns them into SOAP-Faults.
    My first (naive) approach was to do something like this
    Webmethod of the WebserviceBean (
    public ReturnType doWebserviceFunctionality(ParameterType parameter) {
       try {
          beanAinstance.doWork();
          beanBinstance.doMoreWork();
          beanCinstance.doFinalWork();
       catch(SpecificException se) {
       // Map specific exception to ReturnType.specificError
       } catch(Exception e) {  // Bad practice, I know
       // Map general exception to ReturnType.encounteredInternalError
    }In every bean's method as well as in the webservice, I am using Container Managed Transactions (default setting REQUIRED).
    The problem is, that doMoreWork() uses an entity with a versioning attribute (@version) in order to support optimistic locking. This works quite well, however, whenever an OptimisticLockException is thrown, it does not get caught in WebserviceBean and is passed on to the client. I think this is because the complete transaction is commited at the end of doWebserviceFunctionality. I am aware that I could use EntityManager's flush method in bean B in order to catch the OptimisticLockException. However, I am trying not to interfere with transaction commiting as far as possible and furthermore, I am not quite sure, if there could be other exceptions (for example JPA exceptions) which could get thrown in a similar way and are passed on to the client.
    Does anybody know a possibility to solve this problem? Do I have to use bean managed transactions? Or mark doWebserviceFunctionality as not part of an transaction and put the bean method calls in a seperate method which commits at the end and allows me to handle exceptions? Or is there a completely different approach necessary?
    Any help is appreciated, thanks in advance,
    BKelso

    Hello to all,
    thanks again for your suggestions which proved to be valueable hints!
    I'll tell you so far the lessons I've learned.
    1) Using interceptors (suggested by malrawi)
    This would have been a very nice solution, however, it turns out that the problem remains - the exception thrown by the container after the commit is not caught. Here is the code I used
        @AroundInvoke
        public Object convertException(InvocationContext ctx) throws InternalErrorException {
          try {
             return ctx.proceed();
          } catch(Exception e) {
             // just tried to throw another exception here in order to see if it works
             throw new InternalErrorException("Internal Error occured. Please try again later.");
        }2) Using bean managed transcations (suggested by zsom)
    This worked out as I thought it would. The webservice starts and commits the transactions, beanA, B and C can still keep the default CMT Attribute. However, when ut.rollback() is called, an IllegalStateException is always thrown. Don't know why so far, the transaction seems to be set back anyways.
    @Resource SessionContext ctx;
    public ReturnType doWebserviceFunctionality(ParameterType parameter) {
    UserTransaction ut = null;
          ut = ctx.getUserTransaction();
          try {
             ut.begin();
             ut.commit();
         } catch (Exception e) {
             try {
                ut.rollback();
                // map exception...
             } catch (Exception rollbackException) {
                // also map exception to return type
    }3) Upon the suggestion by malrawi using NotSupported transaction attribute in order to force a commit, I came up with the following: The webservice method uses this transaction attribute while simply calling another local stateless session bean (beanZ), which does the actual work (calling beans A, B, C). It uses the default transaction attribute (Required). This works out perfectly at the cost of an additional bean. But maybe you see some other downsides? Essentially it's the same as the idea with the internalMethod made public and using another transaction attribute.
    // web service class
    public ReturnType doWebserviceFunctionality(ParameterType parameter) {
       try {
            beanZinstance.doActualWork();
       catch(Exception e) {  // Map exceptions here
    // bean Z class
    public ReturnType doActualWork(ParameterType parameter) {
           beanAinstance.doWork();
          beanBinstance.doMoreWork();
          beanCinstance.doFinalWork();
    }4) Using soaphandler (suggested by zsom)
    This seems like a good idea but it is also the most complicated and a bit more low-level. But I will check it out in more detail, I think.
    Thanks again for your efforts and I still appreciate any feedback!
    BobKelso

  • Uncaught exception while executing web dynpro application

    Hello all,
    I'm getting run time error exception CX_WDR_RR_EXCEPTION not caught when running web dynpro application
    The error analysis shows the custom component is locked by the user and cannot be regenerated. But I checked in SM12 for locked entries and component was not listed there.
    The program terminated in the include LWDR_RUNTIME_REPOSITORYF12
    Please suggest how to resolve the isse?
    Regards,
    anil

    Hi Anil,
    I'm also getting the same run time error. When I try to activate the Web Dynpro component, I'm getting error that the component is locked by the user.
    Please let me know how you solved this issue.
    Thanks in advance for the quick response.
    Thanks & Regards,
    Jay

  • How to use an ABAP exception as a web service fault

    Hi experts,
    I have created a web service out of an ABAP function module, using the SAP standard wizard for web services. The ABAP function has some exceptions defined. Now the question is: How can I "translate" these ABAP function exceptions to web service faults?
    In the WSDL file I can see that the web service defines the faults, but they are not part of the web service operation (in the WSDL file). So when I load the definition into JDeveloper, the faults are not recognized.
    Any ideas what I am missing here?
    Thanks in advance!
    Kind regards, Matthias

    Exceptions from SAP function module are not translated as web service fault message, this is a standard behaviour due among other to the fact that exceptions are not "in line" with the definition a web service fault message.
    Usually when you want to use a standard SAP function module and expose it as web service, you need to "wrap-it" into a new Z function module.
    In that new function module you must capture the exceptions and convert them into a web service fault message structure (usually containing error type, text and number)
    Karim

  • Uncaught exception when executing web dynpro application

    Hello all,
    I'm getting run time error exception CX_WDR_RR_EXCEPTION not caught when running web dynpro application
    The error analysis shows the custom component is locked by the user and cannot be regenerated. But I checked in SM12 for locked entries and component was not listed there.
    The program terminated in the include LWDR_RUNTIME_REPOSITORYF12
    Please suggest how to resolve the isse?
    Regards,
    anil

    Dear Anil,
    What type of WD is it ?
    Do you get the error when you trigger some sort of event ?
    Is it when first launch the WD ?
    Can you be more specific and share with us the code where this is dumping ?
    You can also check this link
    [CX_WDR_RT_EXCEPTION' - How to catch it?;
    Kind Regards
    /Ricardo Quintas

  • How do you prevent your hyperlinks in Keynote 6.5. from changing?

    Dear Apple community,
    Since I installed the new Keynote version 6.5. I have troubles creating hyperlinks. Whenever I link an email-address or URL to a text object or shape the link gets changed afterwards. For example:
    Display: [email protected]
    Email: [email protected]
    will be changed automatically in:
    Display: [email protected]
    Email: [email protected] (this mail address can be any random email address that has been mentioned in another link in the document before)
    Things I tried:
    - Press enter first
    - Create a new text field instead of copying another textfield in which I simply change the link
    I suspect this to be a bug, but it could be I am doing something wrong. Can anyone help me out? Thank you very much in advance!

    Im not aware of any previous concerns with hyperlinks in 6.5, hyperlinks work perfectly all our systems
    Try resetting Keynote:   delete Keynote using a removal tool Mac App Cleaner is free, (you must use a removal tool for this tequnique to work)
    empty trash
    install from Mac App Store

  • How to trap a java exception where java is invoked from a .bat file

    Hi,
    Jgurus,
    i have problem here, i am invoking a batch file from my java class using the command Runtime.getRuntime().exec(batch1.bat) now this file is setting some classpath and invoking batch2.bat with some parameters and this batch2.bat is invoking java command which is throwing a exception now i want trap this execption.
    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    :::::code for batch1.bat:::::
    set classpath=d:\com\tech\sun\tcc.jar;d:\com\tech\sun\parameter.jar
    set TMMHOME=d:\TMM
    cd %TMMHome%\bin
    testcom.bat -i D:/test1.xml -o D:/testresult.xml
    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    :::::code for testcom.bat:::::
    %_JAVACMD% -classpath %LOCALCLASSPATH% com.tech.sun.parameter
    now when java is invoked it throws an error :
    Exception in thread "main" com.tech.test.runner.RunnerException: com
    .snstech.testcompanion.runner.RunnerException: com.snstech.testcompanion.runner.
    RunnerException: The specified constructor for aclass does not exist
    pls. help me in this issue .....
    thanks in advance
    Abhijit
    [email protected]

    >
    You can read the output stream of that other process.
    Process p = Runtime.getRuntime().exec(...);
    InputStream theOutput = p.getInputStream();
    // or you may want the p.getErrorStream()
    // now read and parse theOutputAre you sure that will work from an exe in a batch file? It was my understanding that a batch file runs in a shell (a process.) When an executable runs in the batch file a seperate process is created. Thus I would expect that it would have its own stdin/out/err in that case.
    However, in newer versions of windows the following will redirect the output to files (there is NO space between the '2' and the '>')
    java.exe MyClass >out.txt 2>err.txt
    This will work if the above does not (and if your version of windows supports it.)

  • How do I prevent iCloud putting all my daughters photos (from her iPad mini) onto my iPad?

    Her mini has my Apple ID for apps and purchases but her own Apple ID for anything free like iMessage and FaceTime, game centre etc. I've just noticed that all her photos are in my photo stream, guessing it happened through iCloud. I really don't want all her stuff clogging up my iPad and definitely don't want her getting ALL my stuff - although it is helpful to be able to have music on both and share apps etc.
    How do I fix it please?

    Photostream is different that iCloud backup, so whether or not it's backed up yet has nothign to do with you seeing her photos.
    If her the photos are appearing in your photo stream, then you two are sharing an iCloud account for sharing photos.
    On your daughter's device in Settings --> iCloud --> Account, which account is listed? It should be your daughter's AppleID (iCloud account). Photos should be turned on.
    On your device, in the same location, your account should be listed.
    There are multiple places where AppleIDs can be used, so be sure you're looking in the place I described above.

  • How do I prevent the plug-in update checker page from opening

    I updated all the plug-ins for which updates were available including Adobe Reader 10. But, every time I open Firefox, the update checker page opens and it indicates Adobe needs to be updated.

    i downloaded and restarted version 18 and still get the plug in page loading it is driving me crazy if i don't find a solution fast i am getting rid of what used to be a great browser hope some one can help me

  • How can I prevent the text in non-resizable contexts from being cut off?

    I have not changed the Application, System, or Dialog fonts from their defaults.  
    This is the result:
    Related:  http://forums.ni.com/t5/LabVIEW/text-in-non-resizable-windows-is-cut-off/td-p/92300

    Medium might be the default for high ppi monitors but it is very different from the original windows settings that were used when 96 ppi was standard. Of course for a high resolution FullHD resolution on a 15 inch screen you end up with a ppi of around 150ppi.
    This article explains somewhat the history of where 96ppi came from and that it was in fact already a convinient misrepresentation of the real values. The problem is partly that LabVIEW makes some assumptions about the size of text based on the actual ppi settings while the rendering of the text as done by Windows can be scaled by your control panel adjustment. However the solution is not as trivial as replacing all assumbtions about the size of text by the new values as that won't always give the correct values either and also has quite a lot of other issues such as controls suddenly looking very oversized when moving a front panel from a low ppi device to a high ppi device. Basically the way Windows deals with screen resolution and different ppi makes it simple for Windows to render text on a display without ugly aliasing effects but makes it impossible for an application to both be text and size accurate when rendering windows like a LabVIEW front panel.
     It's also a reason why so called retina displays still don't really work under Windows while IOS has them already for quite some time.
    Ravens Fan, Windows 7 has Normal (100%), Medium (125%) and Larger (150%). To maintain the relative text size on an old CRT for a FullHD 15 inch labtop display one would have to use the Larger setting, which is outright ugly. I personally use Smaller on all my Windows 7 labtops because of issues like the LabVIEW front panels rendering but also many other applications get into trouble otherwise. Yes it looks kind of small on my FullHD 15 inch LCD monitor, but at least it is all consistent.
    Rolf Kalbermatter
    CIT Engineering Netherlands
    a division of Test & Measurement Solutions

  • How do you prevent double clicks on a web site?

    So that customers don't make a duplicate transactions.

    Hope this Helps
    <html>
    <head>
    <SCRIPT language="JavaScript">
    function go(){
    if(document.test.alreadyHitted.value=="N")
    alert(document.test.alreadyHitted.value);
    document.test.action="Sample.jsp";
    document.test.method="POST";
    document.test.alreadyHitted.value="Y";
    document.test.submit();
    alert(document.test.alreadyHitted.value);
    return true;
    else
    alert("You have already done this Transaction");
    </SCRIPT>
    </head>
    <body>
    <form name="test">
    <input type="hidden" name="alreadyHitted" value="N">
    Test
    </form>
    </body>
    </html>
    Regards,
    Firoz Ahammad Shaik
    Polaris Software Labs Limited

  • How do I display, copy, and paste a web address from an open Safari window please?

    I'm new to Mac and Safari. 
    I'm used to being able to just cut and paste a web address from a page I'm on into a discussion, but I don't see an address bar to copy from.
    I'm sure it is there somewhere, which toolbar or keyboard shortcut am I missing, please?
    Gordon_M

    Hello:
    While Safari is open, go to the menu bar>view.  Select "customize toolbar."  You may use the default, or customize it to yoru liking.
    Then when you wish to copy an address, you simply click on the favicon and go to edit>copy.
    Barry

Maybe you are looking for

  • IPhoto trouble on Macbook Pro. Please help!

    I have reached max capacity on memory on my macbook. Bought an external hard drive to back up all of my pictures and to clear up room on my computer. I did this two days ago. Since then, I have not been able to open iPhoto. When I do, I get this mess

  • Cisco PI 2.0 Configuration Archive fails Nexus 5000

    Hi, we recently upgraded Cisco PI from 1.3 to 2.0 and i thought this problem would be solved but it did not. I have added two Nexus 5548 switches that i can monitor and configure through Cisco Prime but im not able to read the running och startup con

  • Save iGrid as image

    Greetings, Does anyone know how to save an iGrid as an image? We have Chart action block which takes Query Template and Display Template (only Charts) as inputs and saves that chart as image with the help of Image Saver. But for iGrid how to do this?

  • Photo Stream Videos-Shared folders

    Why can't I view a video on iCloud.com that is in a shared photo stream? Why can't I email a video to myself that is in a shared photo stream?

  • Connect to unathenticated email server?

    How do you connect an iphone 4s to an unathenticated email server?  iphone defaults require some form of authentication (password, etc), but my POP3 email host does not use authentication on their server.  Currently no work email on my new work phone