New server and/or CA certificate for connection from custom authentication

Question

We are running Access Manager version 72005Q4 in the Sun ONE Web Server 6.1SP5 B06/23/2005 container with java build 1.5.0_07-b03. I run a custom authentication module which checks sessions against our university single sign on system which is CAS (from Yale/Jasig). The checks are essentially https calls. All this has been working well for us for the last couple of years.
I would like to migrate the certificate used on the university CAS system from a Verisign certificate to a wildcard certificate issued by the IPS CA in spain -- these are in most browsers but are not in the standard batch of cacerts CA's -- and are free for .edu domains.
My other java based authentication plugins (Blackboard, custom apps etc) have worked fine once I import the certificate into the cacerts for the java container, but I'm missing something (obvious probably) about importing this certificate so that my amserver custom authentication module can connect to the CAS server once the CAS server is using the new certificate.
Could anyone provide guidance on where I need to import this server certificate (or preferably the IPS CA) in order to allow the custom authentication module to work properly? I assume this same problem has been solved by people wishing to connect from the amserver to services with self signed certificates. For some reason I'm finding the debugging unexpectedly difficult, I'll outline some of those details below.
Relevant things I've tried so far:
Import both the server cert and the IPS CA into the cacerts of the java container identified in the web server server.xml /usr/jdk/entsys-j2se.
Import the IPS CA into the web server cert8 style db via the web admin server.
The debugging has surprised me a bit, as I'm not getting an error that is explicitly SSL related error. It almost seems like the URLConnection object ends up using a HttpURLConnection rather than an HttpsURLConnection and never gives me a cert error, rather a connection refused since there is no non SSL service running on CAS. The same code pointed to the server running the verisign cert works as expected.
Part of the stack:
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: java.net.ConnectException: Connection refused
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at java.net.PlainSocketImpl.socketConnect(Native Method)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at java.net.Socket.connect(Socket.java:516)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at java.net.Socket.connect(Socket.java:466)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at sun.net.www.http.HttpClient.openServer(HttpClient.java:365)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at sun.net.www.http.HttpClient.openServer(HttpClient.java:477)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at sun.net.www.http.HttpClient.<init>(HttpClient.java:214)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at sun.net.www.http.HttpClient.New(HttpClient.java:287)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at sun.net.www.http.HttpClient.New(HttpClient.java:311)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at sun.net.www.protocol.http.HttpURLConnection.setNewClient(HttpURLConnection.java:489)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at sun.net.www.protocol.http.HttpURLConnection.setNewClient(HttpURLConnection.java:477)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:422)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:937)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at edu.yale.its.tp.cas.util.SecureURL.retrieve(Unknown Source)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at edu.yale.its.tp.cas.client.ServiceTicketValidator.validate(Unknown Source)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at edu.fsu.ucs.authentication.providers.CASAMLoginModule.process(CASAMLoginModule.java:86)
[28/Mar/2008:17:21:54] warning (25335): CORE3283: stderr: at com.sun.identity.authentication.spi.AMLoginModule.wrapProcess(AMLoginModule.java:729)
The relevent bit of code from the SecureURL.retrieve looks as follows:
URL u = new URL(url);
if (!u.getProtocol().equals("https"))
throw new IOException("only 'https' URLs are valid for this method");
URLConnection uc = u.openConnection();
uc.setRequestProperty("Connection", "close");
r = new BufferedReader(new InputStreamReader(uc.getInputStream()));
String line;
StringBuffer buf = new StringBuffer();
while ((line = r.readLine()) != null)
buf.append(line + "\n");
return buf.toString();
} finally { ...
The fact that this same code in other authentication modules running outside the amserver (in other web containers as well, tomcat and resin for example) running java 1.5 works fine with the new CA, as well as with self signed certs that I've imported into the appropriate cacerts file leads me to believe that I'm either importing the certificate into the wrong store, or that there is some additional step needed for the amserver in the Sun Web container.
Thank you very much for any insights and help,
Ethan

Answer 1

I thought since this has had a fair number of views I would give an update.
I have been able to confirm that the custom authentication module is using the cert8 db defined in the AMConfig property com.iplanet.am.admin.cli.certdb.dir as documented. I do seem to have a problem using the certificate to make outgoing connections, even though the certificate verifies correctly for use as a server certificate. This is likely a question for a different forum, but just to show what I'm looking at:
root@jbc1 providers#/usr/sfw/bin/certutil -V -n "FSU Wildcard Certificate" -d /opt/SUNWwbsvr/alias -P https-jbc1.ucs.fsu.edu-jbc1- -u V
certutil: certificate is valid
root@jbc1 providers#/usr/sfw/bin/certutil -V -n "FSU Wildcard Certificate" -d /opt/SUNWwbsvr/alias -P https-jbc1.ucs.fsu.edu-jbc1- -u C
certutil: certificate is invalid: Certificate type not approved for application.
root@jbc1 providers#/usr/sfw/bin/certutil -M -n "FSU Wildcard Certificate" -d /opt/SUNWwbsvr/alias -P https-jbc1.ucs.fsu.edu-jbc1- -t uP,uP,uP
root@jbc1 providers#/usr/sfw/bin/certutil -V -l -n "FSU Wildcard Certificate" -d /opt/SUNWwbsvr/alias -P https-jbc1.ucs.fsu.edu-jbc1- -u C
FSU Wildcard Certificate : Certificate type not approved for application.
So it could be that I don't understand how to use the certutiil to get the permissions I want, or it could be that using the same certificate for both server and client functions is not supported -- though you can see why this would be a common case with wildcard certificates.
BTW for those interested, it did seem to be the case that when the certificate failure occurred that the attempt was then made by the URLConnection to bind to port 80 in cleartext even though the URL was clearly https. I'm sure this was just an attempt to help out misformed URL, but it seemed that the URLConnection implementation in the amserver would swapped traffic over cleartext if that port had been open on the server I was making the https connection to; that seems dangerous to me, I would not have wanted it to quietly work that way exposing sensitive information to the network.
This was why I was getting back a connection refused instead of a certificate exception. The URLConnection implementation used by the amserver is defined by java.protocol.handler.pkgs=com.iplanet.services.comm argument passwd to the JVM, and I imagine this is done because the amserver pre-dates the inclusion of the sun.net.www.protocol handlers, but I don't know, there maybe reasons why the amserver wants it own handler. I only noticed that this is what was going on when I as casting the httpsURLConnection objects to other types trying to diagnose the certificate problem. I would be interested in hearing if anyone knows if there is a reason not to use sun.net.www.protocol with the amserver.
After switching to the sun.net.www.protocol handler I was able to get my certificate errors rather than the "Connection Refused" which is what lead me to the above questions about certutil.