[WLS 9.2] Intermittent performance glitch creating InitialContext

Question

Hi,
We are experiencing a peculiar problem with our WLS 9.2 installations: the time taken to create an InitialContext on a remote host has extremely variable performance.
The setup is that we have a gateway server running WLS 9.2 on Solaris 10 that is sends JMS messages to a number of application servers (all running WLS 9.2/Solaris 10). This is done, in the usual fashion, by creating an InitialContext from which we can lookup the ConnectionFactory and Queue. The gateway server is very sensitive to delays in sending the JMS message and any message that cannot be delivered within a specified time triggers a failover scenario and the message is routed to another server.
Investigation of the what was causing the failover to trigger showed that it is that the creating of the InitialContext that occassionally takes significantly longer than we expected. We set up a simple test in isolation to our system that schedules a task every 30 seconds that only creates an InitialContext and times how long it takes. Here are the stats from two sample runs:
Run 1
Mean: 13 ms
Average: 55.54 ms
Min: 11ms
Max: 5086 ms (5 s)
Sample size: 1566
Run 2
Mean: 22 ms
Avgerage: 139.7 ms
Min: 14 ms
Max: 11444 ms (11 s)
Sample size: 1733
Interestingly, when we run this same test from a Windows-based WLS 9.2 installation, the time reported by the test is 0 ms. I can only assume that there is an underlying
We are working around the problem by creating a pool of message dispatchers that already have an initialised InitialContext but we still run into this problem when the pool needs to grow and additional dispatchers are created.
A smaller scale test on using a Windows-based gateway connection to the same Solaris-based application server did not experience the sample problem.
Any ideas?
The code used for the test:
public class InitialContextTest implements Runnable {
    private static final Logger LOG = Logger.getAnonymousLogger();
    private Hashtable<String, String> _environment = new Hashtable<String, String>();
    public void run() {
        final long start = System.currentTimeMillis();
        weblogic.security.Security.runAs(new Subject(), new PrivilegedAction<Void>() {
            public Void run() {
                InitialContext context = null;
                try {
                    context = new InitialContext(_environment);
                } catch (NamingException e) {
                    LOG.log(Level.SEVERE, "test: Cannot create initial context", e);
                } finally {
                    try {
                        if (context != null) {
                            LOG.log(Level.INFO, "test: Closing initial context");
                            context.close();
                    } catch (NamingException e) {
                        LOG.log(Level.SEVERE, "test: Cannot close initial context", e);
                return null;
        final long end = System.currentTimeMillis();
        LOG.log(Level.INFO, "test: t=" + (end - start));
    public void setEnvironment(Map<String, String> environment) {
        _environment.putAll(environment);
}Edited by: Marcus Edwards on Oct 16, 2009 5:16 PM
remove standard deviation figures at they are invalid

Answer 1

Hi,
I don't know what the issue is, but I imagine there could be any number of causes. Overwhelmed DNS servers, network issues, or a even a very long GC. Or there could be a competing process somewhere local or remote that consuming 100% CPU intermittently (which would freeze out or slow other work).
In case you don't already know:
1 - Contexts are thread safe.
2 -There's usually no need to have multiple contexts if they all resolve to the same remote JNDI host, or even the same cluster (usually no performance benefit).
3 - Multiple WebLogic Java contexts on a local JMS that all connect to the same remote JVM will all multiplex over the same socket. When a new context is created, WebLogic sets up a new network connection, checks to see if the local JVM already is talking to the same server (by comparing data returned from the new connection to data from current connections), and, if so, tears down the new network connection and routes traffic for the new Context over the old pre-existing connection.
4 - As you've already discovered, its best practice to cache contexts. In addition, it's best practice to cache JNDI lookup results, as well as JMS connections, sessions, producer, and consumers.
5 - The test you posted below has a memory leak: it should call context.close() when its done with the context.
Tom