Using a custom SSLSocketFactory for imqbrokerd on startup for SSLJMS

Question

I want to startup imqbrokerd with my OWN custom SSLSocketFactory (to enable decrypting a password to send over as plaintext, so as to NOT store the password in plaintext on the filesystem).
I have tried to use this without success:
imqbrokerd -vmargs -Dssl.ServerSocketFactory.provider=MySSLServerSocketFactory -vmargs -Dssl.SocketFactory.provider=MySSLSocketFactory
I want to send an encrypted JMS message (i.e. SSL / TLS) from a client to a server (JMS topic/Queue) by using a trust/key store file without storing a plaintext password anywhere on the filesystem. I'm using the glassfish OpenMQ (i.e. imq) ver. 4.3.
I have a keystore and truststore combination with passwords for each of the stores. I was able to start up the IMQ Server, and create a simple Java MQ Client that will send JMS messages to the topic/queue with no problem. It seems to work just fine using JMS & SSL-JMS. My dilema is this: I want to be able to start up an SSL connection on the server using the truststore/keystore WITHOUT passing in the password as plain text (i.e. using a simple custom encryption algorithm).
I was able to work around this on the JMS Client by writing my own custom SSLSocketFactory (which decrypts the password stored on disk and sends it on, in memory, unencrypted to allow a client to connect to the server).
I did this by modifying:
*{jre_home}/lib/security/java.security*
file and putting in this line:
ssl.SocketFactory.provider=MySSLSocketFactory
My last problem to solve is how to get the IMQ Server to start up using an encrypted password (stored in a passfile properties file) and then unencrypt it, which will then allow me to open up the keystore and trustore to establish an SSL/TLS connection.
I found that there is a java.security file that will let you put in your own SSLSocket (this was how I got the client to work).
Example:
ssl.SocketFactory.provider=mypackage.MySSLSocketFactory
ssl.ServerSocketFactory.provider=mypackage.MySSLServerSocketFactory
However, these socket factories don't seem to get called when starting up the imq server (i.e. imqbrokerd.exe) for some reason?
I think it is because it is using the default SunJSSE security provider which seems to block the loading of my custom factory Classes?
Does anyone know how to solve this problem? Can I write a custom SSL SocketFactory, or do I have to write a custom Java Security provider or something like that?
Thanks for any ideas you may have.

Answer 1

I found the solution:
The source code for IMQ and J2EE is available to download.
You can then customize the code to do whatever you want (i.e. read in a
password in some encrypted format using an algorithm of your choice and
then unencrypting it before passing it on).
You can easily do this for the KEYSTORE / TRUSTSTORE password files by writing your own Java security provider.
Here are the steps for that:
*1. Write a custom java security provider.*
public class YOUR_PROVIDER_Provider extends java.security.Provider
your code here ...
public class YOUR_PROVIDER_Provider extends Provider
public YOUR_PROVIDER_Provider()
super("YOUR_PROVIDER_Provider", 1.0, "A Custom Security provider");
String className = YOUR_PROVIDER_Provider.class.getName();
put("KeyStore.JKS", className);
2. then,
Need an entry in the java.security file like this:
security.provider.1=YOUR_PROVIDER_Provider
The above order MUST be 1 in order to override the Sun default security provider
*@see java.security java.security file located in: {$JRE_HOME}\lib\security\*
**3. write your own JavaKeyStore* (you can download Sun's java source code to use as a baseline.
I would recommend using this one:
sun.security.provider.JavaKeyStore
If you change the package name to a custom one, you will need this (due to "package protect" accessor of this class)
sun.security.provider.KeyProtector
In this class:
sun.security.provider.JavaKeyStore
Modify these methods:
public Key engineGetKey(String alias, char[] password)
throws NoSuchAlgorithmException, UnrecoverableKeyException{
// ... your code
public void engineSetKeyEntry(String alias, Key key, char[] password,
Certificate[] chain)
throws KeyStoreException{
// ... your code
public void engineStore(OutputStream stream, char[] password)
throws IOException, NoSuchAlgorithmException, CertificateException
// ... your code
public void engineLoad(InputStream stream, char[] password)
throws IOException, NoSuchAlgorithmException, CertificateException
// ... your code
4. put your custom files inside a .jar file and put this in the java classpath AHEAD of the standard java ones:
e.g.
java -Xbootclasspath/p:YOUR_JAR.jar
you will have to do this inside the IMQ startup script
alternatively you can simply edit the imqbrokerd.conf file to something like this:
ARGS=-vmargs -Xbootclasspath/p:YOUR_JAR.jar:
this file is in: IMQ_HOME/etc/mq/