Streaming to HTTP Response through ResultSet?

Question

I have a cursor which returns a large amount of data (potentially millions of rows, each with 10-20 columns). I would like to efficiently create a CSV file from these results by streaming it to the user upon their demand. In other words, I'm not saving a file to the server, I'm piping the results directly to the Response object of my web/client structure.
Currently, I have a resultset, and a BufferedOutputStream attached to the Response object. But I don't know how to properly get the resultset into an InputStream such that I can write it to the output stream in efficient chunks. My code looks like this so far (work in progress):
          String filename = "testfile.txt";
          response.setHeader("cache-Control", "no-store");
             response.setHeader("pragma", "no-cache");
             response.setDateHeader("Expires", 0);
             response.setContentType("application/download");
             response.setHeader("Content-Disposition","attachment;filename=\""+filename + "\"");
             // Getting output stream.
             BufferedOutputStream os = new BufferedOutputStream(response.getOutputStream());                   
             // Buffer for read operations.
             byte[] buffer = new byte[4072];
             // now start the database operations to get the resultset
             Connection conn = null;
          CallableStatement stmt = null;
          ResultSet rs = null;
          String query = "{?= call get_all_data(?,?)}";
          conn = DBUtil.getConnection();
          conn.setReadOnly(true);
          stmt=conn.prepareCall(query);
          stmt.registerOutParameter(1,java.sql.Types.INTEGER);     // return value
          stmt.registerOutParameter(2,oracle.jdbc.OracleTypes.CURSOR);     // CURSOR
          stmt.registerOutParameter(3,java.sql.Types.VARCHAR);     // return message
          stmt.execute();              
          rs = (ResultSet)stmt.getObject(2);
          // here's where I'm unclear
          // how do I get the resultset to the outputstream in specific sized chunks? 
                // where would I apply the CSV formatting to the columns?
          while (rs.next()) {
               os.write(?????);
          rs.close();
          stmt.close();
                os.flush();Edited by: xaeryan on Apr 11, 2011 12:07 PM

Answer 1

Thank you, you are correct in all your comments; We do expect possibly very large files, and this is only a delivery mechanism.
We prevent duplicate requests of this delivery using a lock object (i.e. "already in progress").
As for chunking, that is what I'm trying to figure out and I assumed that would be in the scope of JDBC. You can "chunk" with an InputStream created from a file, so I was hoping there was some elegant way to do the same on the ResultSet, via some method which can treat the ResultSet as an InputStream, although I don't see any striking me as obvious.
The following is a working example we have used for chunking based on reading from a file. Really all I want to do is replace the file with a ResultSet instead:
                  response.setHeader("cache-Control", "no-store");
                  response.setHeader("pragma", "no-cache");
                  response.setDateHeader("Expires", 0);
                  response.setContentType("application/download");
                  response.setHeader("Content-Disposition","attachment;filename=\""+filename + "\"");
                  FileInputStream fis = new FileInputStream(new File(server_path+"/"+server_filename));
                  BufferedInputStream bis = new BufferedInputStream(fis);
                  DataInputStream dis = new DataInputStream(bis);     
                  // Getting output stream.
                  BufferedOutputStream out = new BufferedOutputStream(response.getOutputStream());                   
                  // Buffer for read operations.
                  byte[] buffer = new byte[4072];
                  // dis.available() returns 0 if the file does not have more lines.
                  while (dis.available() != 0)
                        // Read data from input stream ...
                        int length = dis.read(buffer);   // if somehow dis was an inputstream from the resultset?
                        // ... and write data to output stream (response).
                        out.write(buffer, 0, length);
                  // Dispose all input resources after using them.
                  fis.close();
                  bis.close();
                  dis.close();
                  // Flushing output stream. Do not close it. Controller/Servlet is
                  // responsible of it.
                  out.flush();