Generation of temporary files by TUXEDO

Question

We have a customer that has written their own implementation of a TMQFORWARD
server to drain a TUXEDO queue. This server regularly polls a queue and
drains all the messages from it. If there are no messages it has nothing to
do. Our customer however observed that, without any messages to be drained
from the queue, TUXEDO still creates temporary files in /var/temp and that
these files become larger and larger over time. In addition the memory usage
of the server process grows as well. The customer is running TUXEDO 6.5 on
HP-UX 11. Output of the tusc command is presented below. I have the
following questions:
1) Why are the msgsnd() and msgrcv() called if there's nothing to do?
2) Why does TUXEDO try to access a temporary file and then create another
one if there's nothing to do?
3) Does the BEA implementation of TMQFORWARD take any special precautions to
avoid the unnecessary creation of temporary files? Maybe our customer's
implementation is missing some essential element?
In addition a more general question: under what circumstances does TUXEDO
create temporary files apart from coping with large messages? Given the
amount of messages in the catalogues about temporary files, it is my
suspicion that TUXEDO uses temporary files in many situations - but these
are not documented.
Here's the tusc output:
( Attached to process 2803 ("SyncDeliveryServer -C dom=ENV_PR3 -g 9 -i 3021
-u rhino -U /") [32-bit] )
sigtimedwait(0x7bff4660, NULL, 0x7bff4680)
................................... [sleeping]
sigtimedwait(0x7bff4660, NULL, 0x7bff4680)
................................... ERR#11 EAGAIN
time(NULL)
................................................................... =
1057590637
time(NULL)
................................................................... =
1057590637
msgsnd(7132, 0x401e9b58, 500, IPC_NOWAIT|0)
.................................. = 0
sigsetstatemask(NULL, NULL, 119675188)
....................................... = 0
msgrcv(7132, " \0\02 \0\0\0H \0\01bdc\0\0\0\0".., 115996, -1073741824, 0)
... = 500
sigprocmask(0x400778b0, NULL, 0x400778dc)
.................................... = 0
time(NULL)
................................................................... =
1057590637
access("/var/tmp", X_OK|W_OK)
................................................ = 0
getpid()
..................................................................... = 2803
(1)
access("/var/tmp/TUXANEa02803", F_OK)
........................................ ERR#2 ENOENT
lstat("/var/tmp/TUXANEa02803", 0x7bff5948)
................................... ERR#2 ENOENT
open("/var/tmp/TUXANEa028031", O_WRONLY|O_CREAT|O_TRUNC, 0666)
............... = 6
write(6, " \0\02 \0\0\0H \0\01 T \0\0\0\0".., 69520)
........................ = 69520
close(6)
..................................................................... = 0
time(NULL)
................................................................... =
1057590637
msgsnd(12628, 0x400b7d30, 297, IPC_NOWAIT|0)
................................. = 0
msgrcv(7132, "0 \0\0( \0\0\0H \0\01bdc\0\0\0\0".., 115996, 805306408, 0)
..... = 297
open("/var/tmp/TUXFNEa028021", O_RDONLY, 0)
.................................. = 6
fstat(6, 0x7bff5400)
......................................................... = 0
read(6, "0 \0\0( \0\0\0H \0\01bdc\0\0\0\0".., 69512)
......................... = 69512
close(6)
..................................................................... = 0
unlink("/var/tmp/TUXFNEa028021")
............................................. = 0
time(NULL)
................................................................... =
1057590637
time(NULL)
................................................................... =
1057590637
msgsnd(11035, 0x401e9b58, 552, IPC_NOWAIT|0)
................................. = 0
msgrcv(7132, "0 \0\0) \0\0\0H \0\01bdc\0\0\0\0".., 115996, 805306409, 0)
..... = 248
time(NULL)
................................................................... =
1057590637
sigtimedwait(0x7bff4660, NULL, 0x7bff4680)
................................... [sleeping]
Regards,
Winfried Scheulderman

Answer 1

One of the issues I have with a tpacall polling server is that for starters, the
service name has to be unique across all server instances, and secondly advertising
a service means that it can be called by any other server or client. By using
threads you have no advertised services for a periodic job, don't have to worry
about creating a unique service name and dynamically advertising it, and you don't
have to do tpacalls and place dummy messages on the servers IPC queue. A nice
way to do it is to spawn thread(s) in tpsvrinit, one thread per job. Each thread
sleeps for a configurable amount of time and then does it's job. Before doing
the job it locks a mutex to prevent jobs happening concurrently. In tpsvrdone
you lock the mutex to make sure all jobs are complete and then shutdown the threads.
Of course the tpacall way does work and is sufficient for most purposes, but
I prefer the threaded way.
"Winfried Scheulderman" <[email protected]> wrote:
>
Anthony,
Thanks for the information.
Further investigation from our side revealed that the problem is caused
by incorrect
buffer allocation and release when calling tpdequeue() from within the
polling
service.
This causes the buffer to grow continuously.
After some time the buffer becomes too big for the IPC queue used by
the server
and we see the temporary files appear.
These grow as well and may become several Mb's in size, slowing down
the server,
of course.
The solution is simply to correct the buffer allocation and release calls
(tpalloc(),
tpfree()).
With regard to a polling server: I assume that your suggestion, to spawn
a thread
will, work and that you have experience with it.
However, using tpacall() without a data buffer should work fine as well.
I have seen several locations where this mechanism is used.
It works satisfactorily for these customers and they have polling servers
that
run continuously under heavy load until e.g. a TUXEDO reboot. No problem.
Apparently this shows once more that there are multiple ways to Rome.
Cheers, Winfried
"Anthony Fryer" <[email protected]> wrote:
Your customer wrote a server that periodically polls a tuxedo queue.
I'm going
to guess that to do the polling, they do a tpacall with the TPNOREPLY
flags set
to a service contained within the server, and this service is wherethe
queue
is checked for messages. If this is the case, then they probably pass
an input
buffer to the tpacall method. This input buffer normally gets placed
on the
IPC queue of the server, but if the input buffer is too big, then it
is written
to a temporary file in /var/tmp. The input buffer would have to befreed
after
the tpacall, otherwise it would cause a memory leak. Internally, tuxedo
also
allocates memory to hold the input buffers pulled off the ipc queues,
and keeps
an array of these for faster processing which can cause the tuxedo server
size
to grow. I would ask your customer if they have implemented polling
using tpacalls,
and to check the size of the buffers if they have done this. A better
way to
implement a polling server is to spawn a thread that just sleeps and
checks the
queue periodically.
"Winfried Scheulderman" <[email protected]> wrote:
We have a customer that has written their own implementation of a TMQFORWARD
server to drain a TUXEDO queue. This server regularly polls a queueand
drains all the messages from it. If there are no messages it has nothing
to
do. Our customer however observed that, without any messages to be
drained
from the queue, TUXEDO still creates temporary files in /var/temp and
that
these files become larger and larger over time. In addition the memory
usage
of the server process grows as well. The customer is running TUXEDO6.5
on
HP-UX 11. Output of the tusc command is presented below. I have the
following questions:
1) Why are the msgsnd() and msgrcv() called if there's nothing to do?
2) Why does TUXEDO try to access a temporary file and then create another
one if there's nothing to do?
3) Does the BEA implementation of TMQFORWARD take any special precautions
to
avoid the unnecessary creation of temporary files? Maybe our customer's
implementation is missing some essential element?
In addition a more general question: under what circumstances does
TUXEDO
create temporary files apart from coping with large messages? Giventhe
amount of messages in the catalogues about temporary files, it is my
suspicion that TUXEDO uses temporary files in many situations - butthese
are not documented.
Here's the tusc output:
( Attached to process 2803 ("SyncDeliveryServer -C dom=ENV_PR3 -g 9-i
3021
-u rhino -U /") [32-bit] )
sigtimedwait(0x7bff4660, NULL, 0x7bff4680)
.................................... [sleeping]
sigtimedwait(0x7bff4660, NULL, 0x7bff4680)
.................................... ERR#11 EAGAIN
time(NULL)
=
1057590637
time(NULL)
=
1057590637
msgsnd(7132, 0x401e9b58, 500, IPC_NOWAIT|0)
................................... = 0
sigsetstatemask(NULL, NULL, 119675188)
........................................ = 0
msgrcv(7132, " \0\02 \0\0\0H \0\01bdc\0\0\0\0".., 115996, -1073741824,
0)
.... = 500
sigprocmask(0x400778b0, NULL, 0x400778dc)
..................................... = 0
time(NULL)
=
1057590637
access("/var/tmp", X_OK|W_OK)
................................................. = 0
getpid()
= 2803
(1)
access("/var/tmp/TUXANEa02803", F_OK)
......................................... ERR#2 ENOENT
lstat("/var/tmp/TUXANEa02803", 0x7bff5948)
.................................... ERR#2 ENOENT
open("/var/tmp/TUXANEa028031", O_WRONLY|O_CREAT|O_TRUNC, 0666)
................ = 6
write(6, " \0\02 \0\0\0H \0\01 T \0\0\0\0".., 69520)
......................... = 69520
close(6)
= 0
time(NULL)
=
1057590637
msgsnd(12628, 0x400b7d30, 297, IPC_NOWAIT|0)
.................................. = 0
msgrcv(7132, "0 \0\0( \0\0\0H \0\01bdc\0\0\0\0".., 115996, 805306408,
0)
...... = 297
open("/var/tmp/TUXFNEa028021", O_RDONLY, 0)
................................... = 6
fstat(6, 0x7bff5400)
.......................................................... = 0
read(6, "0 \0\0( \0\0\0H \0\01bdc\0\0\0\0".., 69512)
.......................... = 69512
close(6)
= 0
unlink("/var/tmp/TUXFNEa028021")
.............................................. = 0
time(NULL)
=
1057590637
time(NULL)
=
1057590637
msgsnd(11035, 0x401e9b58, 552, IPC_NOWAIT|0)
.................................. = 0
msgrcv(7132, "0 \0\0) \0\0\0H \0\01bdc\0\0\0\0".., 115996, 805306409,
0)
...... = 248
time(NULL)
=
1057590637
sigtimedwait(0x7bff4660, NULL, 0x7bff4680)
.................................... [sleeping]
Regards,
Winfried Scheulderman