How do I process a large number of rows using ADO?

Question

I need to iterate through a table with about 12 million records and process each row individually.    The project I'm doing cannot be resolved with a simple UPDATE statement.
My concern is that when I perform the initial query that so much data will be returned that I'll simply crash.
Ideally I would get to a row, perform my operation, then go to the next row ... and so on and so on.
I am using ADO / C++

Answer 1

I suggest you simply use the default fast-forward read-only (firehose) cursor to read the data. This will stream data from SQL Server to your application and client memory usage will be limited to the internal API buffers without resorting to paging. 
I ran a quick test of this technique using ADO classic and the C# code below and it ran in under 3 minutes (35 seconds without the file processing) on my Surface Pro against a remote SQL Server.  I would expect C++ to be a significantly faster
since it won't incur the COM interop penalty.  The same test with SqlClient ran in under 10 seconds.
static void test()
var sw = Stopwatch.StartNew();
Console.WriteLine(DateTime.Now.ToString("HH:mm:ss.fff"));
object recordCount;
var adoConnection = new ADODB.Connection();
adoConnection.Open(@"Provider=SQLNCLI11.1;Server=serverName;Database=MarketData;Integrated Security=SSPI");
var outfile = new StreamWriter(@"C:\temp\MarketData.txt");
var adoRs = adoConnection.Execute("SELECT TOP(1200000) Symbol, TradeTimestamp, HighPrice, LowPrice, OpenPrice, ClosePrice, Volume FROM dbo.OneMinuteQuote;", out recordCount);
while(!adoRs.EOF)
outfile.WriteLine("{0},{1},{2},{3},{4},{5},",
(string)adoRs.Fields[0].Value.ToString(),
((DateTime)adoRs.Fields[1].Value).ToString(),
((Decimal)adoRs.Fields[2].Value).ToString(),
((Decimal)adoRs.Fields[3].Value).ToString(),
((Decimal)adoRs.Fields[4].Value).ToString(),
((Decimal)adoRs.Fields[5].Value).ToString());
adoRs.MoveNext();
adoRs.Close();
adoConnection.Close();
outfile.Close();
sw.Stop();
Console.WriteLine(DateTime.Now.ToString("HH:mm:ss.fff"));
Console.WriteLine(DateTime.Now.ToString(sw.Elapsed.ToString()));
Dan Guzman, SQL Server MVP, http://www.dbdelta.com