ALUI Gateway Not Returning Scripts for Subset of Users

Question

We have a problem where the ALUI gateway is not returning some .NET scripts for a subset of users. We have the ALUI 6.5 portal and our using the .NET accelerator 3.1.
The situation is that this subset of users request one of our portal pages via https, which then reaches through our firewall to our remote server which is running the .NET portlet. The .NET page is served and returned to the users correctly and quickly, but this particular subset of users do not see the result rendered in their browsers for about 3 minutes. A view html source in the browser, as well as tools like Fiddler, show the page is indeed in the browser, but it is stuck trying to request some .NET scripts, and only displays the page when those requests timeout.
The .NET scripts that are problems are both WebResource.axd and ScriptResource.axd, which in some cases are in our .NET portlets because of the .NET framework itself, but in other cases they are there only because of the ALUI portal itself, when it munges the .NET portlet to handle multiple server forms and validators and such. These .axd scripts are gatewayed so that the client browser requests them through the ALUI gateway, which in turn requests them through our firewall to our remote server -- which always serves these scripts correctly and quickly according to the IIS logs. The problem seems to lie in the ALUI gateway, as it is receiving these scripts correctly and quickly, but it is not returning them to this subset of users. Instead the ALUI gateway seems to be processing for about 3 minutes, and eventually returns an html error page, which of course the client never sees since it is expecting javascript, but we can capture the error page via Fiddler and its just telling us there was a timeout -- the client browser just notes that there is a javascript error.
The really bizarre part is that this only happens for a subset of users, which amounts to about 20% of our users. There are 2 things that delineate these users that we have found so far. First, these users have email addresses that are 27 - 30 characters long, and the email address is our login id. Note that both shorter and longer email addresses are OK, so there is not some limit to email addresses like this might sound like at first. Secondly, these users have to be in a particular branch of our ldap store, which means they are replicated across to the portal in a particular group. We can move these "bad" users into another branch of our ldap store and once they are replicated to the portal then they work fine, and then if we move them back they return to not working. We cannot find any other difference in our ldap branches or in the corresponding ALUI groups, plus its only the ones in that particular branch with the email lengths in that very specific range.
The gatewayed requests for these scripts vary by user since the PTARGS in the gatewayed request include the integer userid, but that does not seem to matter because we can have a "good" user successfully request the script with a "bad" user's id, and we can have a "bad" user fail to successfully request the script with a "good" user's id. That seems to point to maybe the authentication cookie being the differentiating factor that determines whether or not a gatewayed request for one of these script files will succeed or fail. So far we have only seen the problem with these particular .net axd scripts, but that may simply be because we don't have many, if any, other scripts or resources that need to be gatewayed since we usually put resources on our imageserver -- these being different because .NET and/or the ALUI portal puts these references in there for us whether we like it or not. Long-term we can re-architect our .NET portlets to not get have these axd scripts, although as mentioned earlier, we also see the ALUI portal put these axd scripts in our portlets as part of their munging process -- so that is not in our control completely. We do need to test if this subset of users can successfully request other gatewayed resources or not -- this is actually the first time I thought of that test case, so all I can say right now is its axd scripts that we know are problems, but it may or may not be a bigger problem.
One last comment, as we appear to have found a work-around, but it does not make sense at all, and its not our preferred solution, so we still very much believe there is a problem elsewhere -- most likely in the ALUI gateway, but possibly somehow related to authentication that we do not understand. Our work-around that so far seems to work is to make our remote server be accessed via https instead of http -- which matches the way the client browsers call our portal (https). Again that first doesn't make sense, since this is only a problem for a small subset of users -- obviously calling our remote server via http works successfully for all other users, so its not just is a simple case that we must match protocols or it won't work. We also use http successfully for our calls to the remote server for portlets that are Java, although its possible that they don't have any gatewayed resources. But we also would just prefer to not use https for our internal calls in our own network as there is no need for the extra overhead -- and by the way our dev and qa environments do use http even for these .NET portlets and do not have the same problem. What's different in our production environment? The only things that should be different are that we have multiple portal servers and multiple remote servers that are load balanced (not sure that's the right term for how the remote servers are combined) -- and of course we have a firewall between them that does not exist in dev or qa.
So we would very much appreciate any thoughts on this, whether you've seen something like it before, or just have some additional insight into the gateway and/or authentication process that seems to be the issue.
Thanks, Paul Wilson

Answer 1

We've ran into this problem when using the Microsoft ReportViewer control. In our case, we found that the portal gateway malformed the urls containing webresource.axd, so the browser was unable to get the correct address to the files. Note that there are usually multiple links to the axd files, they return different resources depending on the query string they get.
To solve the problem, we ended up with a bit of a hack solution, but it works well. We extracted the resources we needed from the ReportViewer control's assembly using Reflector, and then published them on the image server. The next piece was to override the Render method of the page that hosted the control. In our custom version of Render, we parsed the html of the page, and replaced the contents of the src= elements with pt:images// links. These processed just fine in the portal's transformer, and our resources started showing up.
Our Render looks something like the following code sample. The "HACKReportViewerControlPortalImageGatewayFix" class has all of the code to do the parsing. In this case, it is specific to the report viewer, because it has some special considerations for parsing the urls. My bet if that your code will be quite custom as well. Therefore, I've not included this piece of code. The important piece below is the invocation of MyBase.Render, which tells the page to render all of it's contents. Once that method is done, all of the HTML for the page is in the writer. The ModifyImageTags method then parses the html, doing the necessary replacements. Finally, the modified html is written to the page's writer, so it can be output following the normal .net processes. Also note that when parsing for urls to replace, don't do all of them, just look for the ones containing axd.
(VB.NET)
Protected Overrides Sub Render(ByVal writer As System.Web.UI.HtmlTextWriter)
          Dim fixer As New HACKReportViewerControlPortalImageGatewayFix
          MyBase.Render(fixer.GetWriter)
          writer.Write(fixer.ModifyImageTags())
End Sub
This works great for images. However, if you are dealing with javascripts, I'm not sure if this will work for you - as some .NET controls send different scripts depending on the browser. For example, in IE, you get more buttons on the toolbar for the ReportViewer, so you get more javascript too. When using FF, you get less buttons, and less script. We didn't have a problem with the scripts, so we haven't needed to solve this one.
As for timing, this type of solution doesn't take much to put together. You are really just doing some string parsing and replacements. If you are a regex ninja, it's probably even easier. We had our solution working in a day or two.
An added benefit to this solution is that you are putting less bytes through the portal's gateway, and sending that traffic to the image server instead.