Windows machines can't join domain after 10.5.4 upgrade

Question

Howdy folks,
I have a ticket open with Apple on this but am posting here in hopes that someone might have an idea for me.
I upgraded our Mac OS X Server 10.5.3 to 10.5.4 on Sunday, and this morning several users reported that their PCs running Windows XP SP2 were unable to login to the Windows domain hosted on this machine. It's the primary domain controller for the Windows users.
One thing to note is that I had to reinstall the server completely because the 10.5.4 patcher crashed, creating all kinds of mayhem. I did a fresh install of OS X Server 10.5 and immediately applied the 10.5.4 combo updater to it. I had to restore the Open Directory from an archival copy, and the SMB was created fresh. Not sure why but the SMB services weren't preserved by the Server settings export command in Server Admin.
I thought unbinding the PC from the Windows domain and then rebinding it with a new name would help, but I've been completely unable to add older computers to the domain, even after removing the old computer records first.
I've got a reproducible failure mode for this problem on a Windows XP virtual machine running on VMware Fusion on my Mac. Here's the method I've been using to create the failure:
1. Change Windows XP System name to something new that doesn't already have a computer record on the Mac OS X Server and reboot.
2. After the reboot, run "NewSID" program on Windows to globally change my Windows machine's SID to a new, random value, and reboot again.
3. Attempt to use the Network ID wizard in the Windows Control Panel to re-add the machine to the domain under a new name so there's no conflict with any old computer records floating around in Open Directory. After it prompts me to enter the username, password and domain name for a user who's authorized to add machines to the domain, I get a dialog box that displays this error:
"Your computer could not be joined to the domain because the following error has occurred:
An internal error occurred."
Not too informative.
Here are the error messages I see in /var/log/samba/log.smbd (searching for the new computer name in the search field):
netbios connect: name1=BIGMAC name2=JEFFVM6
netbios connect: local=bigmac remote=jeffvm6, name type = 0
opendirectorysamsearchname gave -14136 [eDSRecordNotFound]: no dsRecTypeStandard:Computers record for account 'JEFFVM6$'
odssam_getgrnam gave -14136 [eDSRecordNotFound]: no dsRecTypeStandard:Groups record for 'JEFFVM6$'!
opendirectorysamsearchname gave -14136 [eDSRecordNotFound]: no dsRecTypeStandard:Computers record for account 'JEFFVM6$'
kDSStdAuthNewUser was successful for account "jeffvm6$"
At that point it's impossible to join the computer to the domain no matter what. The most puzzling thing is that SOME of our users were able to login without any problems whatever. The ones that were either physically off or somewhere else when the 10.5.4 upgrade was applied are the only PCs that seem to be having problems.
Any help at all is appreciated. I suspect this is some kind of a SID conflict because the SMB server had to be recreated from scratch, but have no idea how to fix the client, the server, or both to make the computer account creation process work.

Answer 1

The problem is fixed.
The issue boils down to an argument between the Open Directory server on bigmac (the OS X Server machine) and the SMB server on bigmac. The crucial information I needed to solve this problem was located here: http://www.radiotope.com/node/61
The Open Directory database had to be restored from a backup following this weekend's problematic upgrade, and it had a different value for the SID for the Windows domain than the one used by the SMB server software itself. Even stranger was that the Open Directory database actually had the wrong domain name! The It was listed as "BIGMAC" in Open Directory, even though it was set to the correct Windows domain name in the SMB server.
The solution was to demote the SMB server from a Primary Domain Controller to a Standalone Machine, and then repromote it. Although I changed no values in the settings, and did not modify the plist containing the SID in the Open Directory via the Inspector in Workgroup Admin, after the SMB PDC was repromoted, the SIDs and the domain names in Open Directory and the SMB config agreed with each othe. Now new machines can join the domain and users can login just as they did before. No client-side modifications are necessary.
Hope this is helpful to someone else. It was quite the hair-pulling experience for a while there.
Jeff Kirk