Images (w/correct meta data) are in catalog and on disk, but LR 5.7 considers them new on Import

Question

For reasons explained below, I want to try to re-import all my images into LR and hope that none/few are in fact considered new and are imported.  Yet, for some folders, LR is apparently unable to detect that my source images are already in the catalog, and are on disk, and that the source file meta data matches what LR knows about the images.  When I click an image in LR and Show in Finder, I do see the imported image on disk.  I can edit the image in the Develop module.  So, it seems good, but all is not well.   Sorry for the long post here, but I wanted to provide as much info as I could, as I am really seeking your help, which I'd very much appreciate.
Here are some screen shots that illustrate the problem:
Finder contents of the original images
LR folder hierarchy
an image as seen in LR
Finder content of external LR copy of images
import showing 10 "new" photos
The original images ... (I'm not sure why the file date is April 2001 but the actual image date is January 2011; I may have just used the wrong date on the folder name?)
The LR folder hierarchy ...
An image as seen in LR ...
The external folder containing the images in the LR library
But on import of the original source folder, LR sees 10 "new" photos ...
I tried "Synchronize Folder ..." on this particular folder, and it simply hangs half-way through as seen in the screen shot below.   IS THIS AN LR BUG?   This is really odd, since "Synchronize Folder ..." on the top-level folder completes quickly.
I have a spreadsheet of of the EXIF data for the original files and those created by LR.  (I extracted this info using the excellent and free pyExifToolGui graphical frontend for the command line tool ExifTool by Phil Harvey.)   Almost all of the Exif data is the same, but LR has added some additional info to the files after import, including (of course) keywords.  However, I would not have expected the differences I found to enter into the duplicate detection scheme.  (I didn't see a way to attach the spreadsheet to this posting as it's not an "image".)
I'm running LR 5.7 on a 27" iMac with Yosemite 10.10.2, having used LR since LR2.  I have all my original images (.JPEGs and RAWs of various flavors) on my internal drive on the Mac.   To me this is like saving all my memory cards and never re-using them.   Fortunately, these files are backed up several ways.   I import these images (copying RAWs as DNG) into LR with a renaming scheme that includes the import number, original file creation date and original file name.   There should be one LR folder for each original source file folder, with the identical folder name (usually a place and date).  I store the LR catalog and imported images on an external drive.  Amazingly and unfortunately my external drive failed as did it's twin, same make/size drive that I used as a backup with Carbon Copy Cloner.   I used Data Rescue 4 to recover to a new disk what I thought was almost all of the files on the external drive.
So, I thought all would be well, but, when I tried "Synchronize Folder" using the top-level folder of my catalog, the dialog box appeared saying there were over 1000 "New" photos that had not been imported.  This made be suspicious that I had failed to recover everything.   But actually things are much worse than I thought..   I have these counts of images:
80,0061 files in 217 folders for my original source files (some of these may be (temporary?) copies that I actually don't want to import into LR)
51,780 files in 187 folders on my external drive containing the LR photo library
49,254 images in the top-level folder in the LR catalog (why different from the external file count?)
35,332 images found during import of the top-level folder containing original images
22,560 images found as "new" by LR during import
1,074 "new" images reported by Synchronize Folder ... on the top-level folder in the catalog; different from import count
Clearly things are badly out of sync.   I'd like to be sure I have all my images in LR, but none duplicated.   Thus, I want to try to import the entire library and have LR tell me which photos are new.  I have over 200 folders in LR.  I am now proceeding to try importing each folder, one at a time, to try to reconcile the differences and import the truly missing images.  This will be painful.  And it may not be enough to fully resolve the above discrepancies.
Does anyone have any ideas or suggestions?  I'd really appreciate your help!
Ken

Answer 1

Thanks for being on the case, dj!   As you'll see below, YOU WERE RIGHT!      But I am confused.
    1. Does the same problem exist if you try to import (not synchronize) from that folder? In other words, does import improperly think these are not duplic
YES.  Import improperly thinks they are NOT duplicates, but they are in fact the same image (but apparently not the EXACT SAME bytes on disk!)
    2. According to the documentation, a photo is considered a duplicate "if it has the same, original filename; the same Exif capture date and time; and the same file size."
This is my understanding too.
    3. Can you manually confirm that, for an example photo, that by examining the photo in Lightroom and the photo you are trying to synchronize/import, that these three items are identical?
NO, I CAN'T!  The ORIGINAL file name (in the source folder) is the SAME as it was when I first imported that folder.  That name is used as part of the renaming process using a custom template. However, the file SIZES are different.    Here is the Finder Get Info for both files.  Initially, they appeared to be the same SIZE, 253KB, looking at the summary. But, if you look at the exact byte count, however, the file sizes are DIFFERENT: 252,632 for the original file and 2252,883 for the already-imported file:
This difference alone is enough to indicate why LR does not consider the file a duplicate.
Furthermore, there IS one small difference in the EXIF data regarding dates ... the DateTimeOriginal:
                                                                                                 CreateDate              DateTimeDigitized                    DateTimeOriginal              FileModifyDate                              ModifyDate
ORIGINAL name: P5110178.JPG                                     2001:05:11 15:27:18    2001:05:11 15:27:18-07:00        2001:01:17 11:29:00        2011:01:17 11:29:00-07:00       2005:04:24 14:41:05  
After LR rename:  KRJ_0002_010511_P5110178.JPG    2001:05:11 15:27:18    2001:05:11 15:27:18-07:00        2001:05:11 15:27:18        2011:01:17 11:29:02-07:00       2005:04:24 14:41:05
So ... now I see TWO reasons why LR doesn't consider these duplicates.   Though the file NAME is the same (as original), the file sizes ARE slightly different.  The EXIF "DateTimeOriginal" is DIFFERENT.   Therefore, LR considers them NOT duplicates.
     4a. With regards to the screen captures of your images and operating system folder, I do not see that the filename is the same; I see the file names are different. Is that because you renamed the photos in Lightroom (either during import or afterwards)?
I renamed the file on import using a custom template ...
        4b. Can you show a screen capture of this image that shows the original file name in the Lightroom metadata panel (it appears when the dropdown is set to EXIF and IPTC)?
SO ....
The METADATA shown by LR does NOT include the ORIGINAL file name (but I think I have seen it displayed for other files?).  The File SIZE in the LR metadata panel (246.96 KB) is different from what Finder reports (254 KB).  There are three "date" fields in the LR metadata, and five that I've extracted from the EXIF data.   I'm not sure which EXIF date corresponds to the "Data Time" shown in the LR metadata.
I don't understand how these differences arose.   I did not touch the original file outside LR.   LR is the only program that touches the file it has copied to my external drive during import.  (though it was RECOVERED from a failed disk by Data Rescue 4),
NOW ...
I understand WHY LR considers the files different (but not how they came to be so).  The question now is WHAT DO I DO ABOUT IT?   Is there any tool I can use to adjust the original (or imported) file's SIZE and EXIF data to match the file LR has?  Any way to override or change how LR does duplicate detection?
Thanks so very much, dj.   Any ideas on how to get LR to ignore these (minor) differences would be hugely helpful.