Why is Lightroom 2.4 Thumbnail Rendering SLOOOOOOOOW...?

Question

For some reason Lightroom 2.4 is taking an enormous amount of time to render thumbnail previews and in spite of having my thumbnail cache set to 50 GB, it still re-renders the thumbnails every single time I use the scroll bar.
Every other aspect of Lightroom performs just fine on my machine.  But the library is so slow that it is unusable.  Every time I touch the scroll bar I can see a somewhat darker preview of the thumbnails and then as soon as I stop scrolling, the CPU shoots up to 100% utilization and I watch as the thumbnails brighten and adjust one by one.  30 to 45 seconds later I can click on the thumbnails and navigate around just fine.  Performance is great in fact until I touch the scroll bar again.
The basic question is this.  Why does Lightroom refuse to cache my thumbnails?  And why does it take soooo long to render them when Vista's explorer can render hundreds of thumbnails off even a flash card in seconds?  Even if thumbnail rendering was slow it would be fine if Lightroom would only cache them for me so it doesn't have to redo it every time I scroll back and forth.  Shouldn't there be an option to render thumbnails on import as well as 1:1 previews?
I am running Vista 64 bit with 4GB of RAM with a Gforce 7600GT graphics card.  My library has about 18,000 images in it.  But I've also tried with a brand new small catalog.  I'm shooting and processing Nikon D300 RAW files.
I've done all the standard performance tweaks that everyone talks about:
1.  I render 1:1 previews and never expire them.
2.  I have tried all the various nvidia card tweaks.
3.  I've updated to the latest and greatest nvidia drivers.
4.  I have my catalog on a separate drive from my images, and my thumbnail cache on yet a 3rd drive.
5.  My machine has plenty of RAM, and in fact Lightroom is only using 1.5GB leaving me with well over 1GB remaining.
6.  My machine is not swapping at all.
I've been reading and reading for the past 24 hours about this issue but have yet to come up with an answer to it.  I did find this old thread from over a year ago that describes the EXACT same problem I am experiencing except his problem was with version 2.3:  http://forums.adobe.com/message/1395391#1395391
Does anyone have ANY idea what can be causing this?  As is, I can't use Lightroom under these conditions.
Thanks!

Answer 1

So I spent all evening experimenting with different things in Lightroom to try and improve performance and found 1 solution that worked.  Although I'm not sure why.
I understand that a design decision was made to render thumbs on the fly.  Fine.  But what I couldn't understand was why it took so incredibly long.  I mean the process was literally locking up my entire machine every time I touched the scroll bar.
The ONLY thing I could get to clear up the problem was to select the top level folder in my catalog, and export the entire catalog as a new catalog.  For export I did NOT choose to export the image files or the previews since I only wanted to keep my database (image adjustments, keywords, etc).
Upon opening this newly created catalog, of course everything was slow because there are no previews for anything.  Fine.  So I chose to render standard sized previews for a couple of the folders in the catalog (about 150 photos each).  And then lightroom became extremely fast for browsing and scrolling through thumbnails on those particular folders.
What I can't undertand is why rendering standard sized previews on my old catalog (same photos, same folder) does not improve performance, yet doing it for the new catalog which should be an exact copy DOES improve performance.
The only thing I can think of is that my original catalog must have either been corrupt or there may be some upper limit on how many previews Lightroom can handle gracefully?  Since I had standard previews for my entire catalog before.
I'm going to try and render standard sized previews on my entire new catalog tonight and see if the performance remains snappy.  But something is definitely not right here.