$20 to anyone who can help: (I think) how to send the right cookie info

Question

Yes, we're so befuddled and stumped that we are willing to pay $25 by Paypal or any other method (check, money order) to the first person who provides us with a concrete solution that allows us to read this page through a Java application:
http://s1.amazon.com/exec/varzea/subst/your-account/your-open-marketplace-items.html/104-3907538-7794313
The problem (we think) seems relatively simple: how can we pass the correct cookie to a server? We want to search our merchant web pages on amazon.com (and perform other operations, but for the purposes of this problem, just assume we want to read the above web page). We wrote a variation of a webcrawler which works fine on most web pages. However, the Amazon web pages we want to crawl (i.e., http://s1.amazon.com/exec/varzea/subst/your-account/your-open-marketplace-items.html/104-3907538-7794313) require you to sign in first (otherwise you get redirected to http://s1.amazon.com/exec/varzea/subst/your-account/your-won-zshop-items.html/104-0793551-2976761). So we thought that this meant we had to figure out how to get our webcrawler to login first (we implemented the Java Almanac example for accessing password-protected URLs: http://javaalmanac.com/egs/java.net/Auth.html?l=rel). During the course of testing this out (the code seemed to work, though we still got redirected), we realized that the Amazon web page is not actually performing basic authentication (not asking for username/password), but instead seems (that is, seems to inexperienced us) to be looking for a cookie. We believe this because after we sign in to Amazon, we can access all our merchant web pages just fine without ever needing to log in, even if we turn off the browser (or computer). Also, if we try to access the web page after deleting all cookies, we again get redirected to the page requesting that we sign in.
So we took a look at the Amazon cookie that was created after we signed in to Amazon (printed below), and then implemented the cookie-passing code from the Java Almanac (http://javaalmanac.com/egs/java.net/SendCookie.html). This seemed to have no effect:we still got redirected. We hunted around for other Cookie examples and found achase1's example from a previous forum question (http://forum.java.sun.com/thread.jsp?forum=54&thread=375956), which seemed to add a few HTTPUrlConnection.set's, but this also had no effect--our Java crawler still gets redirected to the page that requests that we sign in first.
So we think that either we are somehow passing the wrong cookie information, or are just missing some critical HttpURLConnection setting or parameter.
So, if you can tell us how to read the Amazon page that seems to require a cookie, and your explanation actually works (that is, we can read the page), we will send you $25 immediately--like so many others on the forum, we're frustrated and lost and need an answer that works!
Here is the Amazon account information (naturally, this is a working dummy account on Amazon, not our actual account, in case you want to test your solution before posting it):
username: [email protected]
password: melville
Here is the cookie that is generated:
session-id
104-3907538-7794313
amazon.com/
1536
3382951936
29569409
1475475408
29568127
session-id-time
1055491200
amazon.com/
1536
3382951936
29569409
1475575408
29568127
ubid-main
430-1017936-7312154
amazon.com/
1536
2916341376
31961269
1482485408
29568127
x-main
Z3yciaQAfpzN?CPFkzeRd8z1U2lWcoap
amazon.com/
1536
2916341376
31961269
2005235408
29568127
Here is the extra-simplified version of our webcrawler, which simply tries to read (and print out) the web page:
import java.net.*;
import java.io.*;
public class PasswordReader {
public static void main(String[] args) throws Exception {
// Try to access the page
try {
     HttpURLConnection m_urlConn;
     URL url = new URL(args[0]);
// Cookie passing code
     m_urlConn=(HttpURLConnection)url.openConnection();
     m_urlConn.setDoOutput(true);
     m_urlConn.setDoInput(true);
     m_urlConn.setUseCaches(false);
     m_urlConn.setRequestMethod("POST");
     // optrional
     m_urlConn.setRequestProperty("User-Agent","Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; H010818)");
     m_urlConn.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
     m_urlConn.setRequestProperty("Cookie" , "session-id=104-3907538-7794313;session-id-time=1055491200;ubid-main=430-1017936-7312154;x-main=Z3yciaQAfpzN?CPFkzeRd8z1U2lWcoap");
     m_urlConn.connect();
// end cookie code
     BufferedReader in = new BufferedReader(
                    new InputStreamReader(
                    url.openStream()));
     String inputLine;
// Read and print out the web page
     while ((inputLine = in.readLine()) != null)
     System.out.println(inputLine);
     in.close();
} catch (MalformedURLException e) {
} catch (IOException e) {
Thank so much to anyone who even tries to help us!! We've been poring through the Sun forums, almanacs, and sample code all week without much evident progress. You'd really be making us very, very happy.
Thank you,
Ogi Ogas
[email protected]

Answer 1

"{[VERSION="0" ; NAME="session_id" ; VALUE="@@33f84622845133891a68ec0dffe9f620" ; DOMAIN="my.asu.edu" ; PATH="/" ; SECURE="false" ; EXPIRES="null"]}"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~The Cookie!
<HTML><HEAD><!--set cookie-->
<SCRIPT language='JavaScript'><!--
document.cookie = "session_id=@@33f84622845133891a68ec0dffe9f620; path=/;";
// Begin JavaScript
if(!document.cookie) {
var agt=navigator.userAgent.toLowerCase();
var is_major = parseInt(navigator.appVersion);
var is_minor = parseFloat(navigator.appVersion);
// Note: Opera and WebTV spoof Navigator.
var is_nav = ((agt.indexOf('mozilla')!=-1) && (agt.indexOf('spoofer')==-1)
&& (agt.indexOf('compatible') == -1) && (agt.indexOf('opera')==-1)
&& (agt.indexOf('webtv')==-1));
var is_nav2 = (is_nav && (is_major == 2));
var is_nav3 = (is_nav && (is_major == 3));
var is_nav4 = (is_nav && (is_major == 4));
var is_nav4up = (is_nav && (is_major >= 4));
var is_navonly = (is_nav && ((agt.indexOf(";nav") != -1) ||
(agt.indexOf("; nav") != -1)) );
var is_nav5 = (is_nav && (is_major == 5));
var is_nav5up = (is_nav && (is_major >= 5));
var is_ie = (agt.indexOf("msie") != -1);
var is_ie3 = (is_ie && (is_major < 4));
var is_ie4 = (is_ie && (is_major == 4) && (agt.indexOf("msie 5.0")==-1) );
var is_ie4up = (is_ie && (is_major >= 4));
var is_ie5 = (is_ie && (is_major == 4) && (agt.indexOf("msie 5.0")!=-1) );
var is_ie5up = (is_ie && !is_ie3 && !is_ie4);
// KNOWN BUG: On AOL4, returns false if IE3 is embedded browser
// or if this is the first browser window opened. Thus the
// variables is_aol, is_aol3, and is_aol4 aren't 100% reliable.
var is_aol = (agt.indexOf("aol") != -1);
var is_aol3 = (is_aol && is_ie3);
var is_aol4 = (is_aol && is_ie4);
var is_opera = (agt.indexOf("opera") != -1);
var is_webtv = (agt.indexOf("webtv") != -1);
var intro_dir = "This installation of Blackboard 5 requires the acceptance of a cookie by your browser software. ";
intro_dir += "The cookie is used to ensure that you <I>and only you</I> are able to access information in the courses, assessments, gradebooks and other features which are appropriate for you. <P>";
intro_dir += "The system has been unable to place the cookie. This may be because cookies are disabled in your browser.<P> To enable cookies in your browser:<ol>";
var nn4dir = "<LI>Select <I>Preferences</I> from your browser's Edit Menu. <LI>Select <I>Advanced</I> from the list in the left-hand pane of the dialog box. ";
nn4dir += "<LI>Under the <I>Cookies</I> box, select either of the first two options ('Accept all cookies' or 'Accept only cookies that get sent back to ";
nn4dir += "the originating server')<LI>Click 'Ok' to close the dialog box. ";
var ie5dir = "<LI>Select <I>Internet Options</I> from your browser's Tools Menu <LI>Select the <I>Security</I> Tab, and click on the 'Custom Level' button. ";
ie5dir += "<LI>Scroll down to the 'Cookies' Section, and select either of the last two options under 'Allow Per-Session Cookies (not stored)' - either 'Enable' or 'Prompt'. ";
ie5dir += "<LI>Click 'Ok' to Close the Security Settings dialog box. ";
ie5dir += "<P><B>NOTE</B> Depending on your institution's set-up of Blackboard 5, you may need to repeat steps 3 & 4 for more than one 'Security Zone'. ";
ie5dir += "<BR>For example, if you are connecting from a computer inside the same firewall or network as the Blackboard 5 machine, you would select the 'Local Intranet Zone'. ";
ie5dir += "<BR>If you are making a connection across the internet from another location, you would select the 'Internet Zone'. <BR>In some cases, you may need to do both.<P>";
ie5dir += "<LI>Click 'Apply' and 'Ok' to close the Internet Options dialog box.";
var ie4dir = "<LI>Select <I>Internet Options</I> from your browser's Tools Menu <LI>Select the <I>Advanced</I> Tab. ";
ie4dir += "<LI>Scroll down to the 'Cookies' Section under 'Security', and select either the first or last option - either 'Prompt before Accepting Cookies' or 'Always Accept Cookies'. ";
ie4dir += "<LI>Click 'Apply' and 'Ok' to close the Internet Options dialog box.";
var browser_dir = "<LI>Please follow your browser's Help instructions for enabling Session (non-stored) cookies that are sent back to the originating server.";
if (is_nav) { browser_dir = nn4dir; }
if (is_ie5up) { browser_dir = ie5dir; }
if (is_ie4) { browser_dir = ie4dir; }
browser_dir += "<LI>Click 'Ok' on this page to return to Blackboard 5.";
document.write("<table border='0' width='100%' cellpadding='0' cellspacing='0'><tr><td align='left' width='40'> </td>");
document.write("<td align='left' width='100%'><b><font face='Arial, Helvetica, sans-serif' size='4'>Browser Cookies Disabled</font></b><hr size=5 noshade></td></tr></table>");
document.write("<table border='0' cellpadding='5' cellspacing='0' width='100%'><tr><td width='20' valign='top'> </td><td width='100%' valign='top'>");
document.write("<font face='Arial, Helvetica, sans-serif' size='2'><b>Browser Cookies Disabled</b></font><br>");
document.write("<font size='2' face='Arial, Helvetica, sans-serif'>"+intro_dir);
document.write(browser_dir);
document.write("</font><br></td></tr><tr><td colspan='6' align='center'><form><input type=button value='Ok' onclick='javascript:history.go(-1)'></td></tr></table></form>");
} else {
var href = document.location.href;
href = href + "?bbatt=Y";
document.location.href = href;
//END JavaScript
//--></SCRIPT>
</HEAD><BODY BGCOLOR='FFFFFF'>
</BODY><HTML>