Active Directory Not Syncing Correctly in ES2

Question

Hello,
We had our Active Directory 2003 synced up using Adobe Livecycle ES.  There would be around 30,000 users that would be synced and this would take around 3 - 4 1/2 minutes to run.  This worked perfectly for us for the past half of a year or so.
Last week we upgraded to ES2 and moved all of our processes over.  We removed ES and did a fresh install of ES2.  Everything seems to be working fine now except the Active Directory isn't syncing properly.  When we run the sync, different numbers of users will be fetched.  Sometimes it's around three thousand, sometimes seven thousand, sometimes ten thousand, but it never seems to get through them all.  In the server log it does say that the directory synchronization completed successfully though even though the number fetched is changing.  We made sure the settings are exactly the same as they were before, and we even tried a few different settings, but it still doesn't get all the users.  For testing purposes, we tried changing the search filter to pick specific people that aren't showing up during the normal sync and it will show up fine, so I'm wondering if there is something stopping it from going all the way through?
We also have another enterprise domain connected which has around 2,000 users on it and have not had this problem with it.
Here are some of the sync statistics from the past few syncs: (The active directory name has been stripped for security purposes).  If you need any more information please feel free to ask.  We would like to have this resolved as soon as possible.
2010-05-30 21:02:51,366 INFO  [com.adobe.idp.um.businesslogic.synch.DomainSynchronizer]
========== Synch Statistics for ============
Total User Fetched - 5633
Total Group Fetched - 0
Total Members Fetched - 0
Total time taken is 110 sec
[100.00%] [100.00%]Domain Synchronizer(2 runs) : Total 110,375 ms, Max 110359 ms, Min 16 ms, Avg 55187 ms
--[99.99%] [99.99%]User and group phase(1 runs) : Total 110,359 ms, Max 110359 ms, Min 110359 ms, Avg 110359 ms
----[95.78%] [95.80%]Users synch from (6 runs) : Total 105,719 ms, Max 19141 ms, Min 14281 ms, Avg 17619 ms
------[1.18%] [1.23%]Provider (31 runs) : Total 1,298 ms, Max 109 ms, Min 31 ms, Avg 41 ms
--[0.01%] [0.01%]Memberhsip phase(1 runs) : Total 16 ms, Max 16 ms, Min 16 ms, Avg 16 ms
-------Persistence Statistics-------
Users ->
added = 8
removed = 2568
updated = 5625
unchanged = 0
renamed = 0
failed = 0
UniqueId changed = 0
Groups ->
added = 0
removed = 0
updated = 0
unchanged = 0
failed = 0
UniqueId changed = 0
Emails ->
added = 8515
removed = 106
unchanged (In changed Principals) = 16784
Group Members ->
added = 0
removed = 0
unchanged = 0
unknown = 0
failed = 0
-------Batch Statistics-------
Successful User Batches = 113
Failed User Batches = 0
Successful Group Batches = 0
Failed Group Batches = 0
Successful Member Batches = 0
Failed Member Batches = 0
======================================
2010-06-02 21:03:43,692 INFO  [com.adobe.idp.um.businesslogic.synch.DomainSynchronizer]
========== Synch Statistics for ============
Total User Fetched - 7140
Total Group Fetched - 0
Total Members Fetched - 0
Total time taken is 165 sec
[100.00%] [100.00%]Domain Synchronizer(2 runs) : Total 164,781 ms, Max 164750 ms, Min 31 ms, Avg 82390 ms
--[99.98%] [99.98%]User and group phase(1 runs) : Total 164,750 ms, Max 164750 ms, Min 164750 ms, Avg 164750 ms
----[96.78%] [96.79%]Users synch from (8 runs) : Total 159,469 ms, Max 26719 ms, Min 3500 ms, Avg 19933 ms
------[1.01%] [1.05%]Provider (42 runs) : Total 1,667 ms, Max 109 ms, Min 15 ms, Avg 39 ms
--[0.02%] [0.02%]Memberhsip phase(1 runs) : Total 31 ms, Max 31 ms, Min 31 ms, Avg 31 ms
-------Persistence Statistics-------
Users ->
added = 8
removed = 5
updated = 7132
unchanged = 0
renamed = 1
failed = 0
UniqueId changed = 0
Groups ->
added = 0
removed = 0
updated = 0
unchanged = 0
failed = 0
UniqueId changed = 0
Emails ->
added = 3340
removed = 105
unchanged (In changed Principals) = 33761
Group Members ->
added = 0
removed = 0
unchanged = 0
unknown = 0
failed = 0
-------Batch Statistics-------
Successful User Batches = 142
Failed User Batches = 1
Successful Group Batches = 0
Failed Group Batches = 0
Successful Member Batches = 0
Failed Member Batches = 0
======================================
2010-06-03 08:56:43,286 INFO  [com.adobe.idp.um.businesslogic.synch.DomainSynchronizer]
========== Synch Statistics for ============
Total User Fetched - 2960
Total Group Fetched - 0
Total Members Fetched - 0
Total time taken is 68 sec
[100.00%] [100.00%]Domain Synchronizer(2 runs) : Total 67,984 ms, Max 67921 ms, Min 63 ms, Avg 33992 ms
--[99.91%] [99.91%]User and group phase(1 runs) : Total 67,921 ms, Max 67921 ms, Min 67921 ms, Avg 67921 ms
----[96.37%] [96.46%]Users synch from (3 runs) : Total 65,516 ms, Max 23016 ms, Min 19766 ms, Avg 21838 ms
------[4.00%] [4.15%]Provider (17 runs) : Total 2,719 ms, Max 844 ms, Min 31 ms, Avg 159 ms
--[0.09%] [0.09%]Memberhsip phase(1 runs) : Total 63 ms, Max 63 ms, Min 63 ms, Avg 63 ms
-------Persistence Statistics-------
Users ->
added = 2
removed = 6632
updated = 2958
unchanged = 0
renamed = 0
failed = 0
UniqueId changed = 0
Groups ->
added = 0
removed = 0
updated = 0
unchanged = 0
failed = 0
UniqueId changed = 0
Emails ->
added = 3
removed = 1
unchanged (In changed Principals) = 10035
Group Members ->
added = 0
removed = 0
unchanged = 0
unknown = 0
failed = 0
-------Batch Statistics-------
Successful User Batches = 60
Failed User Batches = 0
Successful Group Batches = 0
Failed Group Batches = 0
Successful Member Batches = 0
Failed Member Batches = 0
======================================

Answer 1

We do have quite a few that are missing an attribute, specifically:
2010-06-06 21:05:47,579 WARN  [com.adobe.idp.um.businesslogic.synch.LdapHelper] Record [xxxx] is missing required attribute [objectSID] for canonicalName i.e uniqueIdentifier field
This is something that was on our old system as well:
2010-05-25 03:02:35,559 INFO  [com.adobe.idp.um.provider.directoryservices.LDAPDirectoryPrincipalProviderImpl] UserM:: [Thread Hashcode: 3010887] This record is missing a required attribute and cannot be used. Specifically CanonicalName is null. Common Name: xxxx
We have many users in our active directory with just email accounts so that users are able to search for a name and find the email address in outlook.  I have checked through these and they look fine (though there are fewer entries in ES2 since there are fewer users being fetched).
As for the locked users, here is what we received:
2010-06-06 21:05:47,579 INFO  [com.adobe.idp.um.businesslogic.synch.LdapPrincipalProvider] Found [1257] locked users while synching. These users were ignored
This sounds about right for the amount of users that were fetched. 
If you have any more questions or ideas, please let us know.  We would like to have this resolved as soon as possible.  Thanks.