Exporting a PDF from an ATG servlet: IllegalStateException: Response already committed

Question

High-level question
When a custom servlet in my ATG servlet pipeline writes a PDF document into the HTTP response output stream, although the PDF is delivered to the user's client browser successfully, an error is logged on the server: Servlet failed with Exception  java.lang.IllegalStateException: Response already committed".  Why does this occur?
Details
My ATG 10.1.2 application has a requirement to be able to generate and return to the client browser a PDF document when a particular button on the site is clicked.
Although I'm not sure whether or not it's an ATG best practice, the approach that I landed on for this was to insert a new custom servlet, "PdfExportServlet", into the ATG servlet pipeline.  The PdfExportServlet, for requests that it determines are PDF requests and decides to handle, generates the PDF content (using the Jasper Reports engine), and writes the PDF content to the HTTP response output stream.  Here's the relevant portion of the code:
response.setContentType("application/pdf");
// (...Code that generates a JasperPrint object with the PDF content...)
final byte[] pdfBinaryContent = JasperExportManager.exportReportToPdf(jasperPrint);
response.setContentLength(pdfBinaryContent.length);
final ServletOutputStream outputStream = response.getOutputStream();
outputStream.write(pdfBinaryContent);
outputStream.flush();
outputStream.close();
This works fine, and the PDF does display in the client browser.  However, an error is also logged in the application's log file on the server:
<WL-101020> <[ServletContext@1369332778[app:storeFront module:shop path:/shop spec-version:2.5]] Servlet failed with Exception
java.lang.IllegalStateException: Response already committed
    at weblogic.servlet.internal.ServletResponseImpl.objectIfCommitted(ServletResponseImpl.java:1631)
    at weblogic.servlet.internal.ServletResponseImpl.sendError(ServletResponseImpl.java:637)
    at weblogic.servlet.internal.ServletResponseImpl.sendError(ServletResponseImpl.java:602)
    at weblogic.servlet.FileServlet.findSource(FileServlet.java:269)
    at weblogic.servlet.FileServlet.doGetHeadPost(FileServlet.java:191)
    at weblogic.servlet.FileServlet.service(FileServlet.java:173)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
    at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
    at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)
    at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
    at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
    at atg.servlet.GenericFilterService.doFilterChain(GenericFilterService.java:621)
A frustrating thing about this error which had made it difficult to troubleshoot is that it does not occur on my local development PC; this log message only appears when the application is running on one of our pre-production or production servers.  This has made trying to rapidly iterate through multiple possible solutions problematic, as each change to the code requires a fresh deployment to our shared pre-production server, which is time-consuming.
I have tried the following variations on the above code, neither of which has produced a change in the manifestation of the issue (the error message appearing in the log):
Added a call to response.flushBuffer() after the call to outputStream.close(), on the theory that this might serve to notify downstream servlets that the request has already been committed and no further attempts to modify and/or commit the request should be made.
Removed the calls to outputStream.flush() and outputStream.close(), on the theory that some downstream servlet pipeline member (TailPipelineServlet?) might be responsible for closing out the outputStream, and trying to do that prematurely was responsible for the error.
I also so far haven't found anything relevant in the ATG/Oracle documentation that might provide a clue as to what's going on here, or what the general best practice is regarding having a custom servlet in the ATG pipeline that outputs to the response's outputStream.
Any advice regarding the root cause of this error and/or what should be done to fix it would be most welcome!
Thanks,
-Jon

Answer 1

Here's the full stack trace from the log (which includes the PdfExportServlet):
<[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1386007041311> <WL-101020> <[ServletContext@1369332778[app:storeFront module:shop path:/shop spec-version:2.5]] Servlet failed with Exception
java.lang.IllegalStateException: Response already committed
at weblogic.servlet.internal.ServletResponseImpl.objectIfCommitted(ServletResponseImpl.java:1631)
at weblogic.servlet.internal.ServletResponseImpl.sendError(ServletResponseImpl.java:637)
at weblogic.servlet.internal.ServletResponseImpl.sendError(ServletResponseImpl.java:602)
at weblogic.servlet.FileServlet.findSource(FileServlet.java:269)
at weblogic.servlet.FileServlet.doGetHeadPost(FileServlet.java:191)
at weblogic.servlet.FileServlet.service(FileServlet.java:173)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
at atg.servlet.GenericFilterService.doFilterChain(GenericFilterService.java:621)
at atg.servlet.GenericFilterService.handleDoFilter(GenericFilterService.java:484)
at atg.servlet.GenericFilterService.doFilter(GenericFilterService.java:431)
at atg.filter.ConditionalDelegatingFilter.doFilter(ConditionalDelegatingFilter.java:113)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
at atg.service.configuration.ResponseWrappingConfiguration.invokeFilterChainWithPossibleWrapping(ResponseWrappingConfiguration.java:457)
at atg.servlet.pipeline.TailPipelineServlet.service(TailPipelineServlet.java:174)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at com.mycompany.purchase.PdfExportServlet.service(PdfExportServlet.java:73)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.pipeline.DispatcherPipelineServletImpl.service(DispatcherPipelineServletImpl.java:275)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.endeca.assembler.AssemblerPipelineServlet.service(AssemblerPipelineServlet.java:442)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.http.CookieBufferServlet.service(CookieBufferServlet.java:119)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.userprofiling.ExpiredPasswordServlet.service(ExpiredPasswordServlet.java:378)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.pipeline.MimeTyperPipelineServlet.service(MimeTyperPipelineServlet.java:228)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.droplet.DropletEventServlet.service(DropletEventServlet.java:657)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.commerce.order.CommerceCommandServlet.service(CommerceCommandServlet.java:150)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.commerce.promotion.PromotionServlet.service(PromotionServlet.java:213)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.pipeline.PipelineableServletImpl.service(PipelineableServletImpl.java:320)
at com.mycompany.browse.EndecaSessionStateTrackerServlet.service(EndecaSessionStateTrackerServlet.java:97)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.userprofiling.AccessControlServlet.service(AccessControlServlet.java:696)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.sessionsaver.SessionSaverServlet.service(SessionSaverServlet.java:2447)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.userprofiling.PageEventTriggerPipelineServlet.service(PageEventTriggerPipelineServlet.java:191)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.multisite.SiteSessionEventTriggerPipelineServlet.service(SiteSessionEventTriggerPipelineServlet.java:161)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.userprofiling.SessionEventTrigger.service(SessionEventTrigger.java:512)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.userprofiling.ProfilePropertyServlet.service(ProfilePropertyServlet.java:230)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at com.mycompany.servlet.pipeline.ThreadNameServlet.service(ThreadNameServlet.java:89)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.userprofiling.ProfileRequestServlet.service(ProfileRequestServlet.java:460)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.pipeline.DynamoPipelineServlet.service(DynamoPipelineServlet.java:491)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.pipeline.URLArgumentPipelineServlet.service(URLArgumentPipelineServlet.java:302)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.pipeline.PathAuthenticationPipelineServlet.service(PathAuthenticationPipelineServlet.java:392)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.security.ThreadUserBinderServlet.service(ThreadUserBinderServlet.java:113)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.dtm.TransactionPipelineServlet.service(TransactionPipelineServlet.java:234)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.pipeline.SecurityServlet.service(SecurityServlet.java:195)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.multisite.SiteContextPipelineServlet.service(SiteContextPipelineServlet.java:448)
at atg.servlet.pipeline.PipelineableServletImpl.passRequest(PipelineableServletImpl.java:157)
at atg.servlet.pipeline.HeadPipelineServlet.passRequest(HeadPipelineServlet.java:1271)
at atg.servlet.pipeline.HeadPipelineServlet.service(HeadPipelineServlet.java:952)
at atg.servlet.pipeline.PipelineableServletImpl.service(PipelineableServletImpl.java:272)
at atg.filter.dspjsp.PageFilter.innerDoFilter(PageFilter.java:349)
at atg.filter.dspjsp.PageFilter.doFilter(PageFilter.java:208)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3739)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3705)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2282)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2181)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1491)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)