Problems on Linux

Question

I am running redhat 8.
When I start the domain I have an interesting problem.
Issue #1
I run
asadmin start-domain --domain domain1
and I get the following error but both admin-server and server instance are actually running.
Could not start the instance: domain1:admin-server
server failed to start: abnormal subprocess termination
Could not start the instance: domain1:server1
server failed to start: abnormal subprocess termination
Could not start the domain.
Could not start one or more instances in the domain : domain1
I run this to confirm
asadmin list-instances
admin-server <running>
server1 <running>
Issue #2
This one is even more interesting
I run
ps -ef | grep appservd
root 7182 1 0 17:06 ? 00:00:00 appservd -r /opt/sun/appserver7
root 7186 7182 1 17:06 ? 00:00:16 appservd -r /opt/sun/appserver7
root 7260 1 0 17:07 ? 00:00:00 appservd -r /opt/sun/appserver7
root 7264 7260 0 17:07 ? 00:00:10 appservd -r /opt/sun/appserver7
strange no watchdog processes ???
now I take a look at the pid file in the log dir for both admin-server and service instance,
7259
7181
as you might have guess it, these process do not exist
This is a problem as the tools seem to look at these numbers to stop the servers, in this case there is nothing to stop.
So it seems like the issue here is that the watchdog processes are not starting up (not for long at least).
We need a watchdog for a watchdog :-)
Any ideas.
PS I do not have any firewall running on this box (I know that this might be a problem)
cheers
mac

Answer 1

I do have the very same errors on my RedHat 8 installation. However, I do see an 'appservd-wdog' process after I start up the AS7 (see end of posting):
#> ./bin/asadmin start-appserv
Could not start the instance: domain1:admin-server
server failed to start: abnormal subprocess termination
Instance domain1:server1 started
Could not start one or more instances in the domain : domain1
Could not start one or more domains
I have a perfectly working non-RedHat 8 installaion to compare to, and the admin-server's server.log there ends with:
[11/Mar/2003:10:34:05] INFO (23878): HTTP3072: HTTP listener http-listener-1 [http://XXXXXXXX] ready to accept reques
ts
[11/Mar/2003:10:34:05] INFO (23878): CORE3274: successful server startup
[11/Mar/2003:10:34:06] INFO (23932): CORE5053: Application onReady complete.
On the RedHat 8 machine, however:
[11/Mar/2003:12:42:18] INFO ( 873): CORE3274: successful server startup
[11/Mar/2003:12:42:18] SEVERE ( 870): CORE3092: Error receiving response from watchdog
[11/Mar/2003:12:42:18] SEVERE ( 870): CORE3144: Watchdog admin channel has been closed
[11/Mar/2003:12:42:19] INFO ( 927): CORE5053: Application onReady complete.
And unlike in the initial posting from Mac, on my RedHat 8 machine, I do see a watchdog process AND the AS7 is serving the requests properly:
#> ps -efw | grep appservd
root 870 1 0 12:41 ? 00:00:01 appservd -r /opt/SUNWappserver7 -d /opt/SUNWappserver7/var/domains/domain1/admin-ser
root 873 870 13 12:41 ? 00:00:37 appservd -r /opt/SUNWappserver7 -d /opt/SUNWappserver7/var/domains/domain1/admin-ser
root 949 1 0 12:42 ? 00:00:00 ./appservd-wdog -r /opt/SUNWappserver7 -d /opt/SUNWappserver7/var/domains/domain1/se
root 950 949 0 12:42 ? 00:00:01 appservd -r /opt/SUNWappserver7 -d /opt/SUNWappserver7/var/domains/domain1/server1/c
root 952 950 9 12:42 ? 00:00:24 appservd -r /opt/SUNWappserver7 -d /opt/SUNWappserver7/var/domains/domain1/server1/c
In addition to that, I cannot shut down the AS7 anymore using the appserv-stop, instance-stop commands or the GUI interface; the same error messages pop up, the appservd processes (only from the admin-server!) stay alive and the pid files do not get deleted.
#> ./bin/asadmin stop-appserv
Could not stop the instance: domain1:admin-server
server failed to stop: abnormal subprocess termination
Instance domain1:server1 stopped
Could not stop one or more instances in the domain : domain1
Could not stop one or more domains
Is there no way to circle the problem and find out what causes this strange behaviour and why the watchdog doesn't work properly on RedHat 8? Has anyone else experienced this? Any comments are greatly appreciated. Thanks.
Greetings,
Bernd.