Messages linked to perf degradation : 'ignoring message from a previous generation' and 'got heartbeat...but did not find reference'

Question

Hi,
We are facing a problem which implies bad response time from the WL server.
Our configuration is : Sun E4500, Solaris2.6, Oracle 7.3.4, WL oci driver,
no perf pack, Javasoft 1.1.7B_05 vm on server side and IBM 1.1.7B on client
side (W98 clients).
When multiple users connect to the server (let's say from 15 simultaneous
connections) and work on our application, after few mins, the server process
logs and outputs tons of messages like :
Mon Nov 15 16:12:26 GMT 1999:<E> <RJVM> Ignoring message from a previous
generation: JVMMessage from 8803290290807054313C146.100.8.34
to -6807808847876022584C146.100.100.84 with CMD_ONE_WAY, prtNum=5, ack=30,
seqNum=3151
and
Got heartbeat for -6807808847876022584/-1838914476/0/9209 but did not find
reference
Got heartbeat for -6807808847876022584/-1838914476/0/8979 but did not find
reference
This is strangely related to a real degradation of the WL server response
times and sometimes to 'PeerGoneExceptions' thrown to the client (while the
server is still alive...).
This happens either for connections on the LAN or on the WAN (frame relay
network) so I don't think this is network related. As of now, weblogic
support told us not to care about it but after analysing the messages, there
seems to be a real problem here :
What seems to happen is :
- The server considers that the client is dead so it's closing the
connection. Eventually, the DGC cleans up references that were held by this
client.
- The server recieves a query from the 'still alive client' and re-opens the
connection with the SAME id.
- If there is a heartbeat from a DGC cleaned up object, the DGC informs that
it 'did not find reference'.
- It ignores the messages from the previously closed connection and tries to
re-establish everything.
We desperately need some feed back on what is causing the server to close
the connection while the client is still alive. I've seen somewhere in this
list that there are some internal parameters in the WL server
('weblogic.system.idlePeriodUntilTimeout' and
'weblogic.system.periodLength') which indicates the WL server when to
consider a client as 'dead'. This is defaulted to 1 min for the first and to
4 for the second which implies that there must be at least 4 minutes during
which the client doesn't emmit 'heartbeats' for the server to know it is
alive.
few remarks :
1/ the client is not swapping nor GCing too long so the client vm is
available to emmit these heartbeats.
2/ we couldn't reproduce it in our test environement except when making the
client swap for minutes which is not the case in the production env.
3/ this really seems related to multiple clients connected simultaneously to
the server.
Thanks a lot for your help.
Vincent Vandenschrick
Obbisoft.

Answer 1

Michael,
We are running WL Server 404. This is the version we were advised to use for
production. We are not using the performance pack because when we turn it
on, the server behaves correctly for the first 15 mins of use and then,
stops accepting new connections and even closes the existing ones. This
behaviour seems to happen only when the server is used by multiple
concurrent users. So the only "stable" configuration we found to run the
server is Javasoft 1.1.7B (Sunsoft's jdk 1.1.7 produces a core dump after a
few minutes use) + perf pack disabled.
We have support calls open for most of these issues but I thought I would
ask the group to benefit from its experience.
Thanks for your help.
Vincent.
Obbisoft.
Michael Girdley a écrit dans le message
<[email protected]>...
Vincent,
What version of the Server are you running? Why are you not using the
performance pack?
Thanks,
Michael Girdley
BEA Product Manager
Vincent Vandenschrick <[email protected]> wrote in message
news:[email protected]...
Hi,
We are facing a problem which implies bad response time from the WLserver.
Our configuration is : Sun E4500, Solaris2.6, Oracle 7.3.4, WL oci
driver,
no perf pack, Javasoft 1.1.7B_05 vm on server side and IBM 1.1.7B onclient
side (W98 clients).
When multiple users connect to the server (let's say from 15 simultaneous
connections) and work on our application, after few mins, the serverprocess
logs and outputs tons of messages like :
Mon Nov 15 16:12:26 GMT 1999:<E> <RJVM> Ignoring message from a previous
generation: JVMMessage from 8803290290807054313C146.100.8.34
to -6807808847876022584C146.100.100.84 with CMD_ONE_WAY, prtNum=5,
ack=30,
seqNum=3151
and
Got heartbeat for -6807808847876022584/-1838914476/0/9209 but did notfind
reference
Got heartbeat for -6807808847876022584/-1838914476/0/8979 but did notfind
reference
This is strangely related to a real degradation of the WL server response
times and sometimes to 'PeerGoneExceptions' thrown to the client (whilethe
server is still alive...).
This happens either for connections on the LAN or on the WAN (frame relay
network) so I don't think this is network related. As of now, weblogic
support told us not to care about it but after analysing the messages,there
seems to be a real problem here :
What seems to happen is :
- The server considers that the client is dead so it's closing the
connection. Eventually, the DGC cleans up references that were held bythis
client.
- The server recieves a query from the 'still alive client' and re-opensthe
connection with the SAME id.
- If there is a heartbeat from a DGC cleaned up object, the DGC informsthat
it 'did not find reference'.
- It ignores the messages from the previously closed connection and triesto
re-establish everything.
We desperately need some feed back on what is causing the server to close
the connection while the client is still alive. I've seen somewhere inthis
list that there are some internal parameters in the WL server
('weblogic.system.idlePeriodUntilTimeout' and
'weblogic.system.periodLength') which indicates the WL server when to
consider a client as 'dead'. This is defaulted to 1 min for the first andto
4 for the second which implies that there must be at least 4 minutesduring
which the client doesn't emmit 'heartbeats' for the server to know it is
alive.
few remarks :
1/ the client is not swapping nor GCing too long so the client vm is
available to emmit these heartbeats.
2/ we couldn't reproduce it in our test environement except when makingthe
client swap for minutes which is not the case in the production env.
3/ this really seems related to multiple clients connected simultaneouslyto
the server.
Thanks a lot for your help.
Vincent Vandenschrick
Obbisoft.