Missing photos from iPhoto sync in iTunes to iPhone

Question

iTunes is not recognizing the correct number of photos in some of my iPhoto events. Some events are showing as having 0 photos in them, others are showing far fewer photos than the event actually has.
After some digging I found a pattern: the problem only affects "referenced" events, i.e., events where the master copies of the photos are in folders outside of the iPhoto Library. In my case these are mainly older events for which I already had folders before I started using iPhoto. For these events, iPhoto maintains an alias in the library to the original file. The original files are all there, available and editable within iPhoto.
If I edit a photo in one of the affected events, it then appears available for sync in iTunes, presumably because once edited, the modified photo is stored in the iPhoto Library.
To summarize, iTunes does not recognize any iPhoto photos which are stored outside of the iPhoto Library folder (referenced), unless that photo has since been edited in iPhoto.
I'm using Lion 10.7.2 and the latest versions of both iPhoto and iTunes, syncing with an iPhone 4 running iOS 5.0.1.

Answer 1

I've resolved the problem, and it wasn't pretty.
It starts with iPhoto 8 ('09) on my iMac. This library had a bunch of "referenced" photos, ie, photos whos image files were outside of the iPhoto Library folder. In iPhoto 8 this was handled by creating an OS X alias file in the Originals folder within the iPhoto Library folder.
Then I wanted to move my library to my new MacBook Air running Lion and iPhoto 9 ('11). Forgetting that I had referenced files in my library I just copied the iPhoto Library folder from my iMac to my MacBook Air as the Apple support docs say (they don't mention anything about referenced files). Of course the referenced files were not copied to the MacBook, but everything appeared to be OK.
Start iPhoto 9 on my MacBook and it upgrades the library. It can't find the referenced images of course, but there's no warning or anything. iPhoto 9 has a totally different folder and database structure. All the information it needs about where to find an image is now in the Library.apdb SQlite database file in iPhoto Library/Database/apdb/. In that db there's a table called RKMaster which stores the info about each photo. RKMaster has a column called imagePath which is where the full path of the original image file is supposed to go. However, since my referenced images were not available during the iPhoto library upgrade it just set these all to "Masters/<image file name>", and stored the alias information from the iPhoto 8 database in a BLOB column called fileAliasData. When I worked out the referenced files were missing I copied them over to my MacBook, and the iPhoto was miraculously able to find them again because of the information in the fileAliasData column in the database.
However, iTunes and other apps that link to the iPhoto library don't access the iPhoto Library database directly. They use a file called AlbumData.xml in the iPhoto Library folder. iPhoto 9 re-creates this file every time you close iPhoto. Unfortunately, what it uses to build the full image path for putting in the XML file is what's stored in the imagePath column in the database, which by now is basically garbage. iPhoto can see all the files OK, but nothing else can.
The only solution was to painstakingly fix the RKMaster records for all my referenced files so that they pointed to the correct place. I took the opportunity to copy the files to the iPhoto Library folder in the process so that now all my images are in the iPhoto Library folder. This implied setting fileIsReferenced to 0 and fileAliasData to null for those records aswell.
Now I finally have everything working again, but it was a very long and tedious process that can only be done with some knowledge of SQL databases. Apple should really warn people about this potential problem when advising on how to move your iPhoto Library.