Can't bind server to OD, replication broken, users at some sites can't auth

Question

Hi all,
Having a doozy of a problem with our OD at the moment, hopefully someone can help
The setup:
1 OD master and 3 replicas at head office here, all running 10.5.5 (issue also occurred in 10.5.4)
Around a dozen remote AFP & SMB file servers, all are setup as "Connected to a directory system"
Most sites are OK, but we have issues at 2 sites.
Setting each site to "Connected to a directory system" fails at the Directory Utility stage. We try and add the master (or even some of the replicas), put in the diradmin name and password, and attempt to bind, but it responds with an error after a while.
The error states that there is already a computer with this name, and prompts to overwrite. Overwriting fails also.
I did a search and found this:
http://support.apple.com/kb/TS1245
and this:
http://forums.bombich.com/viewtopic.php?t=11834&highlight=lkdc
But neither tips help
Attempting to set up the 2 remote servers as replicas stops at the "Enabling Password Server Replication" stage. I can close out of the assistant at this stage and am left with a "broken" replica, which has 2 out of 3 things running:
LDAP Server: Running
Password Server: Stopped
Kerberos: Running
On the master it says "Password Service Not Found"
So it seems that setting up replicas on a different subnet doesn't work.
I've tried getting my ISP to set up a static route for the VPN tunnel, and this worked for a set of blank test servers with no extra users added to them.
Interestingly, setting a user's password to "Crypt" in WGM allows them to authenticate to the "broken replica" and access their files. Setting their password type to "Open Directory" has no luck at all.
If I jump on the server and try a password check in the terminal, I get:
AFP:
dirt -u username -p password
Call to dsGetRecordList returned count = 1 with Status : eDSNoErr : (0)
Call to checkpw(): Bad Password
path: /LDAPv3/10.10.20.1
Username: username
Password: password
Error : eDSAuthFailed : (-14090)
SMB:
dirt -a nt -u username -p password
Call to dsGetRecordList returned count = 1 with Status : eDSNoErr : (0)
path: /LDAPv3/10.10.20.1
Username: username
Password: password
Good
The master's IP is 10.10.20.1.
Users can sometimes connect via SMB instead of AFP, which is a workaround for now, but I'd like to know why this is happening.
I've tried setting the AFP server's authentication methods to Standard instead of Any or Kerberos, to no effect.
Does anyone know why these servers won't bind and won't replicate, and only allow connections if people use Crypt passwords? Is my Kerberos stuffed?

Answer 1

Well, I tried to demote, then promote my master with no luck. During the upgrade from Tiger to Leopard, the password service cache (or whatever it's called) was somewhat corrupted. My OD Archive failed to restore !! I had to rebuild my entire OD from scratch.
Now I have slightly less errors. My replicas joined up fine. Kerberos passwords get propagated when a password changes, but samba passwords do not. Samba working is essential as 95% of the clients machines are windows boxes. Here are a few quick tests I did:
replica1 root# dirt -m /LDAPv3/127.0.0.1 -u fred
User password:
2009-01-15 17:13:16.919 dirt[4224:10b] password is : <password>
Call to dsGetRecordList returned count = 1 with Status : eDSNoErr : (0)
Username: fred
Password: <password>
Success
replica1 root# dirt -m /LDAPv3/127.0.0.1 -a nt -u fred
User password:
2009-01-15 17:13:23.160 dirt[4233:10b] password is : <password>
Call to dsGetRecordList returned count = 1 with Status : eDSNoErr : (0)
Username: fred
Password: <password>
Error : eDSAuthFailed : (-14090)
On the master this all works fine.
The funny thing is that my diradmin account has not problem on the replica.