Solaris 10 x86 login hanging

Question

We are experiencing some odd behavior with our Solaris 10 machine. We installed it in January and everything was working fine up to about March. Since March we've been seeing an issue with the login hanging. This happens when running command via rsh or ssh (e.g. "ssh [machine] ls -la"). It also occurs if you try and telnet to the machine.
We currently have a script that polls several machines in our environment including this one using "ssh [machine] uname -a" just to make sure the machine is up and running.
About once a week we can't login to the machine with the id we always use, builder. However logging in as root still works. Rebooting fixes the problem every time. Any ideas as to what might be the problem or how to debug it?

Answer 1

I see the following in /var/log/authlog
May 7 10:40:06 zimbo sshd[29122]: [ID 800047 auth.crit] fatal: Timeout before authentication for 9.11.126.223
I've also tried doing an 'ssh -v' for the verbose output and here is what I get.
#ssh -v zimbo ls -la
OpenSSH_4.3p2, OpenSSL 0.9.7g 11 Apr 2005
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Failed dlopen: /usr/krb5/lib/libkrb5.a(libkrb5.a.so): 0509-022 Cannot load module /usr/krb5/lib/libkrb5.a(libkrb5.a.so).
0509-026 System error: A file or directory in the path name does not exist.
debug1: Error loading Kerberos, disabling Kerberos auth.
debug1: Connecting to zimbo [9.11.224.216] port 22.
debug1: Connection established.
debug1: identity file /home/builder/.ssh/id_rsa type 1
debug1: identity file /home/builder/.ssh/id_dsa type 2
debug1: Remote protocol version 2.0, remote software version Sun_SSH_1.1
debug1: no match: Sun_SSH_1.1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_4.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-cbc hmac-md5 none
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host 'zimbo' is known and matches the RSA host key.
debug1: Found key in /home/builder/.ssh/known_hosts:41
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering public key: /home/builder/.ssh/id_rsa
[at this point it just hangs]