Specific Problem with Library Module 1:1 Preview Loading

Question

Hi everyone.  I've spent hours searching Google & forums for specific tips to try but I've found nothing that solved my problem.  I have a very specific problem loading 1:1 previews in Lightroom 3.3 while zoomed to 1:1.
When I initially import a shoot (usually 3000 to 6000 images), I always render 1:1 previews during import.  My understanding is that this activity writes full-sized previews to the "Previews" folder, but I also have observed that .DAT files are created in the "Camera Raw Cache" folder.  I have my Lightroom Previews folder and my Adobe Camera Raw Cache folder on the **same** solid-state hard disk, which is dedicated to Lightroom.
When I browse these newly-imported images in the Library module in Loupe view at 1:1 zoom, it's very zippy and fast to switch between images.  The images load without any message at a lower resolution, and then "snap" to full resolution after 0.25 to 0.5 seconds. No "Loading" message appears.
However, once the previews have "sat around for a while", even if they are still supposedly available in the Previews folder, loading the image at 1:1 in Loupe as described takes 3-5 seconds.  Shortly after going to an image, the message "Loading..." is displayed (it doesn't say where from), and 3-5 seconds later, the full res image appears.
This has nothing to do with changes to image settings requiring re-rendering of the preview.  These are images that I've imported but haven't browsed in a while.  I know that the previews are valid, because I can go to "Library > Previews > Render 1:1 Previews", and the task will complete within 20 or 30 seconds without re-rendering any photos after examining them and apparently finding that there are still valid 1:1 previews for all 4000 of them.  After doing this "re-render," the slow loading behavior is the same as before, because the re-render process always completes without actually updating the previews that it found are already there and presumably decided were still valid.  Discarding previews by "Library > Previews > Discard 1:1 Previews" and then trying to re-render them also does nothing, because LR3 apparently doesn't immediately delete the discarded previews, so they are picked back up again As-Is by the re-rendering process.
If I had to guess at the reason for this behavior based on the symptoms, I would say that it seems like Lightroom is forcing itself to re-render 1:1 previews upon browsing to images that are older than a certain age (very possibly the same "30 days" after which the 1:1 previews are supposed to be discarded).
Now, I've also observed that deleting the DAT files in the Camera Raw Cache folder has no adverse impact on load speed (which is what I would have expected to see if LR3 was looking for and loading images from this cache instead of the Previews folder).
What this means to me is that once the image previews reach this "outdated" state, I can't get fast 1:1 browsing performance in Loupe with any reliable procedure other than manually deleting [in Windows Explorer] the entire Preview folder and then re-rendering the 1:1 previews for this shoot.  This is of course terribly inconvenient when there are recent previews that I want to save.
Note:  I do not have any slowness under any circumstance when browsing at "Zoom to Fit".  This is only a problem when I'm browsing at 1:1 zoom (like when I'm comparing sharpness between a set of photos).
Note:  For images with the slow behavior described, once the image has been loaded once at 1:1 zoom, it's henceforth snappy again when returning to the image (as if Lightroom was once again accepting the Preview file as valid).
So, here's a summary:
Previews (X:\Lightroom Catalog Previews\) and Camera Raw Cache (X:\Lightroom Cache\) are on same, dedicated, fast drive
1:1 previews from recently-imported files load full-res at 1:1 zoom in less than 1/2 second.
Older 1:1 previews take 3-5 seconds to load full-res at 1:1 zoom, even though the previews are supposedly being loaded from still-valid 1:1 previews in the Previews folder.
I'm really tired, so i hope this makes sense.  I don't see how this could be a hardware problem, because I get good performance with recently-imported images.  I'm running LR3.3 on Win 7 Pro x64 with 8 GB of RAM and a 2.4 GHz Intel Core2 Quad CPU.  Never had LR3 beta installed - upgraded to 3.0 from the last 2.X version.  After upgrading, in response to this exact issue, I deleted my entire previews folder and rebuilt, but I continue to have the exact same problem on every shoot that I import.  It's most noticeable on the large shoots, because I end up having to browse at 1:1 zoom to compare a set of similar images.
Only workaround I've found:  in Windows Explorer, manually delete the Previews folder.  Return to Lightroom and rebuild 1:1 previews.  This forces the previews to be rebuilt for all images.
Another possible workaround I am going to experiment with: setting the Previews to "Do Not Discard", in case the defualt "Discard after 30 days" that I've used is causing or contributing to the problem.  If Lightroom is actually discarding the images after 30 days but forgetting that it had discarded them when asked to re-render 1:1's, maybe setting to "Do not discard" will bandage that problem by never discarding them to begin with.
All right, I'm falling asleep at the keyboard.  I'm afraid to add more details as they may become incoherent.  Hope to hear some good news, as fixing this problem would make my LR3 experience so much more pleasant.
Can anyone assist with this problem?
- Tyson

Answer 1

function(){return A.apply(null,[this].concat($A(arguments)))}
Ian Lyons wrote:
The behaviour after previews have been around for a while (i.e. previews for unedited images) is not normal. There have been a few other reports of similar behaviour, but no real clues as to why it happens.
Yes - this is my problem.  Previews should still be valid, and "Library > Previews > Render 1:1 Previews" says the images are still valid, but then Loupe rebuilds them anyways.
function(){return A.apply(null,[this].concat($A(arguments)))}
Ian Lyons wrote:
If everything was working correctly, the setting for discard shouldn't have any effect on whether the previews go stale or not. ...
My understanding is that if everything were working as intended, the "setting for discard" that I was referring to (Edit > Catalog Settings > File Handling > "Automatically Discard 1:1 Previews") should be the sole determinant of whether 1:1 previews must be rebuilt for images with unmodified settings.
For example, if I import an image and render a 1:1 preview, and if I set the "Auto Discard 1:1" setting to "30 Days", and if I make no changes to the image's settings, then I would expect that the image should render quickly (under 1/2 second) at 1:1 zoom in Loupe view for the next 30 days.  On the 31st day, I would expect the image to take longer to load as a new 1:1 preview was rendered and stored, and then I would expect this new preview to be retained for another 30 days.
I would furthermore expect that the 30-day counter would be reset every time the image was re-rendered (i.e. if I go into Develop and change something).
To a certain extent, this is functioning correctly on my system.  When I go to one of these "problem images" that inexplicably load slowly, returning to them subsequently is quick.  I think my problem is just, as Rob suggested, that some part of Lightroom's "brain" has wacky ideas about whether an up-to-date 1:1 preview exists or not.
If I choose to "Never" automatically discard 1:1 previews (which is what I just switched to), then I should "Never" see unmodified images taking 3-5 seconds to "Load" at 1:1 zoom in Loupe.
Since I just nuked my Previews folder again last night and re-rendered 1:1's overnight for a 3000 image shoot, it might be a while before I start encountering this bug again.  I will certainly post any further related experiences here and will keep checking for anyone who has further insight.
function(){return A.apply(null,[this].concat($A(arguments)))}
Ian Lyons wrote:
...we know that Lr3.4 RC fixed the manual discard bug, so, if I were you I would focus any further tests on Lr3.4RC.
Thanks for pointing this out.  I will upgrade to 3.4RC - at least then I will be able to force a rebuild of previews from one shoot without having to manually delete the previews for my entire catalog.  I'll no longer have to throw out all the food in the refrigerator just because something smells funny.
function(){return A.apply(null,[this].concat($A(arguments)))}
Ian Lyons wrote:
The DAT files are created at the same time as Library previews. However, they are only used in Develop module.
Yes, I definitely agree with this (although, the Cache is probably also used while rendering new previews after "Quick Develop" changes in the Library module....).  I posted my observations to this effect just in case someone else wondered the same thing I did.  It is clear that the DAT files get written every time a preview is rendered for RAW file.  Rob, now that I'm thinking with a clearer mind, I think the only thing these Cached files are used for is if you "return" to recently rendered settings in Develop or Quick Develop - then, that particular combination of settings yields a match in the cache, and LR loads the image from the Cache into Previews and onto the screen.  It is logical that the cache wouldn't help us when going to new settings, because the new settings would create a partially or completely different rendering than any other combination of settings.  Adobe may be going to more advanced levels of "reusability" than this, but I'm sure this is the general state of things.
You may want to open back up write access to that Cache folder...
Thanks, Rob & Ian!
So, my question, or bug report, still remains for now - why am I biting into a bag of chips marked "Fresh" and finding them Stale?  And why does it say "Paper Jam" when there is no paper jam?