UWC/CE 6.3 and Access Manager 7.1 SSO sometimes fails (seems like a bug)

Question

PREAMBULA: I started writing this post thinking that our AM SSO setup was at fault in some step. As I was gathering data, checking the doc-links and config files and finally sniffed the servers for HTTP dialogs, I grew pretty sure there's a bug in UWC/CE, AM SDK or Web Server Policy Agent, whatever implements the AM SSO session checking.
In short, as written below, our "sunmail" server can POST a broken cookie to AM server, if the cookie originally contained a "plus" character. The "plus" is replaced by a "space", invalidating the session check. As we know, "+" is often used in URLs to "escape" the space character. Perhaps some URL cleanup routine backfired here. I have double-checked, it is not the reverse proxy on "psam" breaking things. It is "sunmail" (UWC/CE or Policy Agent, don't know for certain) supplying the broken request. On the few occasions when the AM cookie contains no "plus" characters, the SSO works like a charm (also checked by a sniffer). Whenever there is a "plus", it breaks.
Is there some known bug or workaround that matches this description?
Nevertheless, for completeness' sake I kept the description of our setup. Maybe it's at fault after all :)
We have an installation of JCS5 with the latest patches as of early July 2008. And as the subject implies, we have problems with AM SSO in UWC/CE web-interface. I have reported them before, then they seemed fixed (not occuring for several tests in a row), but as time has shown, something wrong is still there.
So I'll try to go into deeper detail now, as we've may have overlooked some nuance... Then again, as my sniffer research below shows, this may be an engine bug and these setup details are irrelevant.
Our setup is split into several Solaris 10 full-root zones hosted on several servers, some of the components are enroute to HA (perhaps we made some mistakes on this part of the way?)
So, we have the following software stack:
1) two MMR Directory Servers (DSEE 6.3 = DSEE 6.2 from JCS5 + 125278-07__DSEE_6.3__x86x64 + 125277-07__DSEE_6.3__x86_sol9 patches) working in zones on two different servers. Except for one time when a manually forced ZFS rollback corrupted one of the server instances, no problems here.
2) two zones with Directory Proxy Servers (6.3, exact versions as above) running at port 389 provide the clients with an illusion that they have a stable Directory Server, even if one of the actual servers is currently rebooting ;)
These DPS zones are hosted on two different servers as well and are primarily used by LDAP clients (JCS components) running in other zones on the same respective servers.
3) A zone with Sun Web Server 7.0U1 and Access Manager 7.1 (+ 126357-01__AM71_x86 patch) and Delegated Admin 6.4-4.01 (from JCS5 + 121582-18__COMMCLI64__x86 patch).
At the moment there is one such zone (named "cos-psam-01.domain.ru" in the logs below), but we expect(-ed) it to become two similar zones as per AM HA setup.
Zones listed in (1-3) use private IP numbers, they belong in our internal DMZ.
Zones listed in (4-5) below use public (routed) IP numbers, they belong in our external DMZ.
4) A zone with Sun Web Server 7.0U1 used primarily as a reverse-proxy server (optionally with a load-balancer libpassthrough.so plugin) successfully used for other hosted projects. One of its configurations now passes connections from an externally routed IP address published as "psam.domain.ru" to "cos-psam-01.domain.ru", per AM HA setup, so HTTP clients believe they work with an Access Manager instance. This zone has a backend interface with a private IP address to communicate with the actual AM instance.
In AM configuration (both LDAP and file-based) we have configured a site ID with the publicly known name and mentioned both names (psam and cos-psam-01) in organization's realm/dns aliases.
5) A zone with the rest of the Sun Java Communications Suite 5, as in Messaging Server 6.3 (6.3-6.03 64-bit: ci-5.0-1.03_solx86_x64__Messaging_Server_6.3-2 + patch 126480-09__MSG63__x86-64), UWC/CE 6.3 (from JCS5 + 122794-17__UWC63-4.01_core__x86), Instant Messaging 7.2 (from JCS5 + 118790-29__IM72__x86-1 + 118787-28__IM72__x86-2), Calendar Server 6.3 (from JCS5 + 121658-28__iCS63__x86). The web-components (UWC/CE, IM, /httpbind) are deployed in a Sun Web Server 7.0U1 as well.
This zone is named "sunmail.domain.ru" and has a routed IP address for direct external access to its servicess.
The AM SDK part is also patched (126357-01__AM71_x86); it points to the load-balancer name ("psam.domain.ru") as an actual AM server.
# imsimta version
Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 64bit)
libimta.so 6.3-6.03 (built 17:15:08, Mar 14 2008; 64bit)
SunOS sunmail 5.10 Generic_127112-07 i86pc i386 i86pc
While setting up this server set we tried to use AM SSO as the user login method, but it works unreliably.
"Unreliably" means that while most of the time entering a correct uid and password in Access Manager login page ("http://psam.domain.ru/amserver/UI/Login") does redirect a user back to "http://sunmail.domain.ru/uwc/auth" along with a new cookie, and the user is redirected again to his or her mailbox, sometimes the user receives the UWC/CE login page. Entering the same uid and password here does log him in, but it breaks the whole point of SSO and only increases the end-user routine required to log in :\
We have also seen the "missing mail tab" problem - if the users point the browser to any hostname different from "sunmail.domain.ru" (i.e. www.mail.domain.ru which is equivalent in DNS), they have only the Address book, Calendar and Options tabs; no webmail. So far this is resolved by Policy Agent forcing The One name of the server.
Here's the configuration we did specifically for AM SSO:
1) in AMConfig.properties of "sunmail" and "cos-psam-01" we set up
com.iplanet.am.cookie.encode=false
am.encryption.pwd=<the same value>
all hostname-related parameters point to "psam.domain.ru"
2) in AMConfig.properties of "cos-psam-01" a number of FQDN equivalence entries are added (so it does not redirect to a server hostname unknown to visitors):
com.sun.identity.server.fqdnMap[publicname-or-ip]=psam.domain.ru
com.sun.identity.server.fqdnMap[cos-psam-01.domain.ru]=cos-psam-01.domain.ru
3) in "msg.conf" on "sunmail" (entries added via configutil):
local.webmail.sso.amcookiename = iPlanetDirectoryPro
local.webmail.sso.amnamingurl = http://psam.domain.ru:80/amserver/namingservice
local.webmail.sso.singlesignoff = yes
local.webmail.sso.uwcenabled = 1
service.http.ipsecurity = no
(perhaps some more options are required? Looking for confirmation about: local.webmail.sso.uwclogouturl local.webmail.sso.uwccontexturi local.webmail.sso.uwchome service.http.allowadminproxy )
4) Configured Web Policy Agent for Sun Web Server, so that users without an AM session are required to get one. Set up per [http://msg.wikidoc.info/index.php/AM_redirection_using_Policy_Agent], except that com.sun.am.policy.agents.config.notenforced_list points to the many names our server can go known by.
5) Updated the logout URL in /opt/SUNWuwc/webmail/main.js:
--- main.js.orig        Sat Jan 26 07:52:09 2008
+++ main.js     Mon Jul 21 01:06:29 2008
@@ -667,7 +667,8 @@
function cleanup() {
   if(laurel)
-      top.window.location =  getUWCHost() + "/base/UWCMain?op=logout"
+//      top.window.location =  getUWCHost() + "/base/UWCMain?op=logout"
+      top.window.location =  "http://sunmail.domain.ru:80/base/UWCMain?op=logout"
   else
       exec('logout', '', 'exit()')
@@ -1707,7 +1708,8 @@
   if(lg) {
         url = document.location.href
         url = url.substr(0,url.indexOf('webmail'))
-        uwcurl = url + 'base/UWCMain?op=logout'        
+//      uwcurl = url + 'base/UWCMain?op=logout'        
+        uwcurl = "http://sunmail.domain.ru:80/base/UWCMain?op=logout"
   exit()
}6) Calendar SSO - per docs...
According to ngrep sniffing,
1) the browser goes to "http://sunmail.domain.ru/uwc/auth" without any cookies
2) receives a redirect and goes to "http://psam.domain.ru/amserver/UI/Login?gotoOnFail=http://sunmail.domain.ru:80/uwc&goto=http%3A%2F%2Fsunmail.domain.ru%3A80%2Fuwc%2Fauth"; sends no cookies either.
3) The first response from the "psam" server (as redirected from "cos-psam-01") sets a few cookies while rendering the login page:
Set-cookie: JSESSIONID=7EF8F2810D2071CA03CFEAE9972735B2; Path=/
Set-cookie: AMAuthCookie=AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1+xTqH7C3I=@AAJTSQACMDI=#; Domain=.domain.ru; Path=/
Set-cookie: amlbcookie=02; Domain=.domain.ru; Path=/
4) The browser requests the login page resources (javascripts, images, etc) using these cookies, as in this header line:
Cookie: JSESSIONID=7EF8F2810D2071CA03CFEAE9972735B2; AMAuthCookie=AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1+xTqH7C3I=@AAJTSQACMDI=#; amlbcookie=02
5) The browser POSTs the login request to "/amserver/UI/Login" and receives a redirection to http://sunmail.domain.ru:80/uwc/auth
Set-cookie: iPlanetDirectoryPro=AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1+xTqH7C3I=@AAJTSQACMDI=#; Domain=.domain.ru; Path=/
Set-cookie: AMAuthCookie=LOGOUT; Domain=.domain.ru; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/
6) The browser requests "http://sunmail.domain.ru/uwc/auth" using the newly set cookie (looks like the old one to me though):
Cookie: amlbcookie=02; iPlanetDirectoryPro=AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1+xTqH7C3I=@AAJTSQACMDI=#
7) The "sunmail" web-server checks the AM session validity with the same "psam.domain.ru". It sends a series of POSTs to /amserver/namingservice:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<RequestSet vers="1.0" svcid="com.iplanet.am.naming" reqid="685">
<Request><![CDATA[
<NamingRequest vers="1.0" reqid="324" sessid="AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1+xTqH7C3I=@AAJTSQACMDI=#">
<GetNamingProfile>
</GetNamingProfile>
</NamingRequest>]]>
</Request>
</RequestSet>(receives a large XML list of different Access Manager configuration parameters and URLs)
...then a double-request to /amserver/sessionservice:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<RequestSet vers="1.0" svcid="Session" reqid="686">
<Request><![CDATA[
<SessionRequest vers="1.0" reqid="678">
<GetSession reset="true">
<SessionID>AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1+xTqH7C3I=@AAJTSQACMDI=#</SessionID>
</GetSession>
</SessionRequest>]]>
</Request>
<Request><![CDATA[
<SessionRequest vers="1.0" reqid="679">
<AddSessionListener>
<URL>http://sunmail.domain.ru:80/UpdateAgentCacheServlet?shortcircuit=false</URL>
<SessionID>AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1+xTqH7C3I=@AAJTSQACMDI=#</SessionID>
</AddSessionListener>
</SessionRequest>]]>
</Request>
</RequestSet>As a result it receives an XML with a lot of user-specific information (the username, LDAP DN, preferred locale, auth module used, etc.)
!!!*** Now, the problem part ***!!!
8) And then "sunmail" POSTs a broken cookie to "psam" (note the space in mid-text, where the "plus" sign was previously). As we know, "+" is often used in URLs to "escape" the space character. Perhaps some URL cleanup routine backfired here.
I have double-checked, it is not the reverse proxy on "psam" breaking things. It is "sunmail" (UWC/CE or Policy Agent, don't know for certain) supplying the broken request. I looked over the large XML responses to the two previous requests, whenever they mention the session cookie value, the "plus" is there.
For the most detail I can provide, I'll even paste the whole HTTP packet:
POST /amserver/sessionservice HTTP/1.1
Proxy-agent: Sun-Java-System-Web-Server/7.0
Cookie: iPlanetDirectoryPro=AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1 xTqH7C3I=@AAJTSQACMDI=#;amlbcookie=null
Content-type: text/xml;charset=UTF-8
Content-length: 336
Cache-control: no-cache
Pragma: no-cache
User-agent: Java/1.5.0_09
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Host: cos-psam-01.domain.ru
Client-ip: 194.xxx.xxx.xxx
Via: 1.1 https-weblb.domain.ru
Connection: keep-alive
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<RequestSet vers="1.0" svcid="session" reqid="258">
<Request><![CDATA[<SessionRequest vers="1.0" reqid="254">
<GetSession reset="true">
<SessionID>AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1 xTqH7C3I=@AAJTSQACMDI=#</SessionID>
</GetSession>
</SessionRequest>]]></Request>
</RequestSet> The server's error response is apparent:
HTTP/1.1 200 OK
Server: Sun-Java-System-Web-Server/7.0
Date: Thu, 31 Jul 2008 05:49:50 GMT
Content-type: text/html
Transfer-encoding: chunked
19b
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResponseSet vers="1.0" svcid="session" reqid="258">
<Response><![CDATA[<SessionResponse vers="1.0" reqid="254">
<GetSession>
<Exception>AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1 xTqH7C3I=@AAJTSQACMDI=# Invalid session ID
AQIC5wM2LY4SfcwuT2ASCrsfO78nXdceEHXeH1 xTqH7C3I=@AAJTSQACMDI=#</Exception>
</GetSession>
</SessionResponse>]]></Response>
</ResponseSet>On the few occasions when the AM cookie contains no "plus" characters, the SSO works like a charm (also checked by a sniffer). Whenever there is a "plus", it breaks.
For reference, here's a working final request-response (one with a good cookie, as received by the load-balancer web-server). Request looks a bit different:
POST /amserver/sessionservice HTTP/1.1
Cookie: iPlanetDirectoryPro=AQIC5wM2LY4Sfcy/5sEzVmuq9z1ggdHOkBDgVFAwfhqvn4U=@AAJTSQACMDI=#;amlbcookie=null
Content-Type: text/xml;charset=UTF-8
Content-Length: 379
Cache-Control: no-cache
Pragma: no-cache
User-Agent: Java/1.5.0_09
Host: psam.domain.ru
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<RequestSet vers="1.0" svcid="session" reqid="281">
<Request><![CDATA[<SessionRequest vers="1.0" reqid="277">
<SetProperty>
<SessionID>AQIC5wM2LY4Sfcy/5sEzVmuq9z1ggdHOkBDgVFAwfhqvn4U=@AAJTSQACMDI=#</SessionID>
<Property name="uwcstatus" value="active"></Property>
</SetProperty>
</SessionRequest>]]></Request>
</RequestSet> ...and the response is OK:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResponseSet vers="1.0" svcid="session" reqid="281">
<Response><![CDATA[<SessionResponse vers="1.0" reqid="277">
<SetProperty>
<OK></OK>
</SetProperty>
</SessionResponse>]]></Response>
</ResponseSet>

Answer 1

There have been a few reports of the same behaviour with other customers - specifically with the handling of the encoding of "+" characters to " ". It relates to how cookie encoding/decoding is performed (as you have already observed).
The solution for these customers was the following:
=> AM server/client side:
Ensure that com.iplanet.am.cookie.encode=false in AMConfig.properties and AMAgent.properties on all systems.
=> AM client (UWC) side:
- Set <property name="encodeCookies" value="false"/> in /var/opt/SUNWuwc/WEB-INF/sun-web.xml. This will prevent UWC from trying to urldecode the cookie it receives and therefore stops it turning the + into a space e.g.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE sun-web-app PUBLIC '-//Sun Microsystems, Inc.//DTD Sun ONE Application Server 7.0 Servlet 2.3//EN' 'file:///net/wajra.india.sun.com/export/share/dtd/sun-web-app_2_3-1.dtd'>
<sun-web-app>
   <property name="encodeCookies" value="false"/>
   <session-config>
      <session-manager/>
   </session-config>
   <jsp-config/>
<property name="allowLinking" value="true" />
</sun-web-app>Regards,
Shane.