Content-Type=application/x-www-form-urlencoded with UTF-8 data which servlet fails to interpret correctly .

Question

My Environment is Weblogic 6.1 SP2 on WIN2K.
I have an HTTP client sending an HTTP request with
Content-Type=application/x-www-form-urlencoded
the data in HTTP request is in UTF-8. Note that HTTP client does not specify charset=UTF-8
as part of the Content-Type.
I have a servlet "VosXmlEvents" processing this HTTP request.
The HTTP request contains data like “Zoë” which the servlet below
interprets as “Zoë” causing me to think that my deployment settings
might be wrong.
My Deployment settings:
In weblogic.xml deployment descriptor I have specified following:
<charset-params>
<!-- I specified this so that I can tell to weblogic that HTTP request data is
UTF-8 so weblogic can successfully convert it to ISO-8859-1 as per Java servlet
spec.-->
<input-charset>
<resource-path>/vos/events/xml/*</resource-path>
<!-- I tried both UTF8 and UTF-8 and got same results-->
<java-charset-name>UTF8</java-charset-name>
</input-charset>
<!-- I specified the the settings below just to be on the safe side, as I have
seen java documentation, newsgroups etc. using slightly different, names for the
same char encoding/canonical names . Java uses MIME preferred names,I think I
dont have to do this but I hope specifying these should not do any harm either,
am I correct in assuming it? (see reference 1 and 2. below for details)-->
<charset-mapping>
<iana-charset-name>ISO_8859-1:1987</iana-charset-name>
<java-charset-name>ISO-8859-1</java-charset-name>
</charset-mapping>
<charset-mapping>
<iana-charset-name>ISO_8859-1:1987</iana-charset-name>
<java-charset-name>ISO8859_1</java-charset-name>
</charset-mapping>
<charset-mapping>
<iana-charset-name>UTF-8</iana-charset-name>
<java-charset-name>UTF-8</java-charset-name>
</charset-mapping>
<charset-mapping>
<iana-charset-name>UTF-8</iana-charset-name>
<java-charset-name>UTF8</java-charset-name>
</charset-mapping>
</charset-params>
My web.xml has:
<servlet>
<servlet-name>VosXmlEvents</servlet-name>
<servlet-class>some class name</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>VosXmlEvents</servlet-name>
<url-pattern>/vos/events/xml/*</url-pattern>
</servlet-mapping>
According to Servlet spec. 2.3 sec. SRV.4.9 Request data encoding, cut pasting
from spec below:
"Currently, many browsers do not send a char encoding qualifier with the Content-Type
header, leaving open the determination of the character encoding for reading HTTP
requests. The default encoding of a request the container uses to create the request
reader and parse POST data must be &#8220;ISO-8859-1&#8221;, if none has been
specified by the client request. However, in order to indicate to the developer
in this case the failure of the client to send a character encoding, the container
returns null from the getCharacterEncoding method.
If the client hasn&#8217;t set character encoding and the request data is encoded
with a different encoding than the default as described above, breakage can occur.
To remedy this situation, a new method setCharacterEncoding(String enc) has been
added to the ServletRequest interface. Developers can override the
character encoding supplied by the container by calling this method. It must be
called prior to parsing any post data or reading any input from the request. Calling
this method once data has been read will not affect the encoding."
Q1. Should not <input-charset> related settings in weblogic.xml for a servlet
helps container actually do some thing like setCharacterEncoding(String enc) on
HTTP Request before it call the servlet?
Q2. If not then that would mean I have to programatically call setCharacterEncoding(String
enc)to correctly interpret my HTTP Request?
Q3. If the answer to Q1 is "yes" then I will assume that getInputStream() or getReader()
methods on the HttpServletRequest will give the chacter in the client encoded
format which in this case is UTF-8, i.e I will get "Zoë" as in this example. Am
I correct here?
Reference:
1. http://edocs.bea.com/wls/docs70/webapp/components.html#139932
2. http://java.sun.com/j2se/1.3/docs/guide/intl/encoding.doc.html

Answer 1

Stefan,
Thanks for the information. I have the following question then.
The Webserver I am interacting with does not recognize user and password provided in the channel. It accepts user and pwd in payload and in application/x-www-form-urlencoded  content type only.
Sample Raw Post that server server requires looks like this
User=yourname&Password=yourpassword&INPUT_XML=%3C%3Fxmlversion%3D%221.0%22standalone%3D%22no%22%3F%3E%3CDELIVERY%3E%3CMESSAGE%3E%0D%0A++++%3CDESTINATION_ADDR%3E%2B447900570205%3C%2FDESTINATION_ADDR%3E%%3C%2FMESSAGE%3E%0D%0A%0D%0A%3C%2FDELIVERY%3E%
What i have done is that constructed the post manually in Java code in un encoded form and expected the HTTP Adapter to do the encoding. Having done this, the HTTP Server is able to scuccessfully parse the xml except for & and < , > chars which err out as invalid XML.
Is there an elegant way of doing above scenario using XML post and standard HTTP Adapter.
Best Regards,
Sudharshan N A