Running long processes

Question

I have a series of pages that perform the following
functions:
1. Query the database and present a list of selectable items
with checkboxes next to each item. The user selects any number of
items to report on.
2. The selected items within the form are sent to a
processing page that generate CFDOCUMENT pdf reports via a series
of queries to the database for each item. The generated reports are
saved to a temporary directory on the server and when the last
report is created, they are zipped up and emailed to the user.
All of this works quite well except that it can be a fairly
long process especially if a fair number of items are selected. I
can override the CF page timeout with the CFSETTING tag but there
is an issue with the browser timing out. I had implemented a
progress bar with a series of CFFLUSH commands to keep the browser
receiving content, eliminating the browser timeout. However, the
page now seems to ignore the flush commands and just displays it
all at the end instead of incrementally. This seemed to happen when
I changed the submitting form to one that posts the form data
instead of calling the processing page with url variables. This was
required when I went to a multiselect option. I'm not sure if this
is the cause but I have tried everything I can to get the CFFLUSH
flushing again to no avail.
Is there a way to get the server to process the request in
background mode? Since the user makes a request and doesn't receive
any meaningful content on the screen after the request (they just
expect the email to be sent), is there any way to make the server
process the request even if the user disconnects or goes onto
another page? This would be ideal. If not, does anyone have
suggestions on getting CFFLUSH to work again. I have tried all the
obrious things like checking for tags that are incompatible with
the CFFLUSH tag and the IE need to have a certain amount of content
sent to it before the CFFLUSH. Simple test pages seem to work.

Answer 1

RE: bpdevlin
I think you're on the right track. Capture and store whatever
data you need (even credentials if necessary) in a new database
table. As a side-note; if you are concerned about the credentials
being in the database, you could consider using encryption.
You could then create a new scheduled process with
<cfschedule>, and have that process run pretty much
immediately (by scheduling the task for a date in the past). That
process would query the database for the data it needs, and then
proceed with its "work". Once it's done "working", it could report
a "done" status back to the database, or simply delete the record
entirely.
The front-end could periodically check for this "done" status
(or for the absence of the record in the database), to know when it
can send the user to the "view report/results" page. There are a
lot of ways to accomplish this; one simple approach I've taken in
the past is to embed a 1-pixel X 1-pixel <iframe> in the page
that runs the "check if done" script over and over again. While
this is happening, the main page/window could show an animated GIF
that simply says "Please wait while we process your request...."
(or something along those lines)
There are a lot of different ways you could approach this,
but I just thought I'd throw a few ideas out there to get your
creative juices flowing :)
Best of luck with it.