Can Web Cache rewrite HTTP_REFERER headers?

Question

Can Web Cache be configured to rewrite the HTTP_REFERER header to replace the Web Cache site address:port with the dispatched application server address:port?
i.e.
If there is a Web Cache site listening on webcache.domain:80 and it is mapped to application servers: appserver1.domain:7778 and appserver2.domain:7778.
And a user requests page webcache.domain/page1, which contains a link to page2.
And then the user opens the link to page2, and Web Cache decides to send the request on to appserver2.
Can Web Cache be configured to change the HTTP_REFERER header from webcache.domain/page1 to appserver2.domain/page1?
This is important to our application, because we want to verify that our processing pages are invoked from our application validation pages (we know this method is not 100% tamper proof).
We would prefer to not have to change our application code to support Web Cache
Thanks for any info or suggestions, -Steve

Answer 1

Thanks Jean
I've added comments to extracts from your post.
Let's say you suddenly get a request for p2, even if
it was from the history of the browser, it would still
contain a referer header of p1, and is no different
from what it would be when the user was going through
the proper steps? Are you saying the p2 generated from
your JavaScript will have a different referer from the
p2 generated from browser history? If so, how are they
differ exactly?The browser history does not include the referer. So p2 requested from the history list does not include a referer although the original link from p1 did.
Without Web Cache, the flow is:
Step 1: a user requests appserver/page1. page1 is added to the browser history. page1 is loaded and executes JavaScript to display page2 in an iFrame. page2 is not added to the browser history (just like style sheets and other embedded requests do not get included in the history). page2 is loaded and executes JavaScript to submit an html form to load page3 into a hidden iFrame and wait for it to load. page3 is added to the browser's history becuase it was submitted; not embedded. Assuming that the request for page1 came from the browser's history, favorites, a link from an e-mail (the usual for us), or was entered manually - there will be no referer for page1. The expected referer for page2 is appserver/page1, and the expected referer for page3 is appserver/page2.
Step2: while processing page3, the application compares the server name in the requested URL (appserver/page3) with the server name from the referer URL (appserver/page2), if the servers match, page3 accesses the database and sends the requested data to the browser
Step3: JavaScript in page2 detects that page3 has loaded, interprets the results from page3, updates the content displayed by page2, and returns control to the browser user
Since this works with your app, how is webcache in
front going to break it? Note that when passing a
request
to the app server, web cache does not modify the
host header in the request.without Web Cache the browser sends requests directly to the appserver, so the request/referer entries seen by the application look like:
appserver/page1 -
appserver/page2 appserver/page1
appserver/page3 appserver/page2
with Web Cache the browser sends requests to webcache, so the request/referer entries seen by the application look like:
appserver/page1 -
appserver/page2 webcache/page1
appserver/page3 webcache/page2
This is because the Web Cache changed the request URL and did not change the referer URL.
So request to webcache.domain/abc will be sent to your
app server as is.I don't think so, doesn't the requested URL change as a result of the Web Cache mapping? i.e. webcache.domain/abc --> appserver.domain/abc
And since we do not change the referer
header, either, whatever works for your app now should
still work for you with webcache in front. This is
assuming you only do comparisons on the 2 hosts to see
if they are the same. Web Cache is changing one (request URL) of the two items (request URL and referer URL) that the application server compares.
The reason has been that we are
a cache, and in principal should be as transparent
as possible, so we try to stay away from app-sever/
apache like functionality.I support that principle 100% - from the application server perspective, it should not need to differentiate between a request directly from a browser, and a request that was issued by the cache on behalf of a browser.
The cache should completely pretend that it is the originating browser. A browser sends the request and the referer URLs. In application flow, a relationship can be inferred/required between the two URLs. Web Cache is changing one without changing the other and breaking the relationship that the application server would normally expect.
But I'm still
not convinced that you need that in webcache. :)If we want to use Web Cache in front of this application, without changing the application, then we need it. :)
Of course, we could change our application, but the more I think about the principles involved, the more I believe that it is the cache that needs changing. However, I will take another look to see if there are other standard headers related to a request that are present with and without Web Cache, and that can be used to reliably compare the server in the request URL with the server in the referer URL. If there are, then I can happily change our application so that it will work with or without Web Cache.
-Steve