Getting Portal pages with utl_http (how-to)

Question

I was so happy when I managed to fetch Portal pages using utl_http I thought I'd share my code.
The example function is supposed to be called through the browser (although probably not directlty),
logged in or not. The function will set the current cookies to support page access and/or
item level security.
My versions are
Oracle Portal Version: 9.0.2.2.22
Oracle9i Enterprise Edition Release 9.0.1.3.1 - Production
PL/SQL Release 9.0.1.3.1 - Production
CORE     9.0.1.2.0     Production
TNS for 32-bit Windows: Version 9.0.1.3.0 - Production
NLSRTL Version 9.0.1.3.0 - Production
Here's the code, hope someone has use of it:
CREATE OR REPLACE FUNCTION GetPortalPage ( psURL in varchar2 ) return varchar2 IS
arrCookies utl_http.cookie_table;
arrNames owa_cookie.vc_arr;
arrValues owa_cookie.vc_arr;
nCount integer;
req utl_http.req;
resp utl_http.resp;
data VARCHAR2(2048);
sHTML varchar2(32767);
BEGIN
utl_http.clear_cookies;
-- get the cookies set in the browser
owa_cookie.get_all(arrNames, arrValues, nCount);
for i in 1..nCount loop
arrCookies(i).name := arrNames(i);
arrCookies(i).value := arrValues(i);
arrCookies(i).domain := 'yourdomain.com';
arrCookies(i).expire := sysdate + 1;
arrCookies(i).path := '/';
arrCookies(i).secure := false;
arrCookies(i).version := 1;
arrCookies(i).comment := null;
end loop;
-- set the cookies so that utl_http uses them as well
utl_http.add_cookies( arrCookies );
req := utl_http.begin_request(psURL, 'GET');
-- without the next two lines Portal will assume you are a
-- mobile device or something :)
utl_http.set_header(req, 'Accept', 'text/plain, text/html');
utl_http.set_header(req, 'User-Agent', 'Mozilla/4.0');
resp := utl_http.get_response( req );
begin
loop
utl_http.read_text(resp, data, 2048);
sHTML := sHTML || data;
end loop;
exception
when utl_http.end_of_body then
-- this error is expected when the whole page is read
null;
end;
utl_http.end_response( resp );
return sHTML;
END GetPortalPage;

Answer 1

I also need to fetch Portal pages using utl_http to manipulate the html code before returning to the browser. Even after applying the tricks from the sample above I'm still unable to forward the cookies returned from the browser to the utl_http request to make it effectively part of the original Portal session the browser uses.
Below is a piece of code from a test case I've prepared, that displays cookies:
CREATE OR REPLACE package http_test is
procedure proc1;
procedure proc2;
end http_test;
CREATE OR REPLACE package body http_test is
procedure show_cookies
is
t_names owa_cookie.vc_arr;
t_vals owa_cookie.vc_arr;
n_num_vals integer;
begin
owa_cookie.get_all(t_names,t_vals,n_num_vals);
htp.p('No of cookies: '||to_char(n_num_vals)||'<br>');
for i in 1..n_num_vals loop
htp.p(t_names(i)||' = '||t_vals(i)||'<br>');
end loop;
end show_cookies;
procedure forward_cookies
is
t_names owa_cookie.vc_arr;
t_vals owa_cookie.vc_arr;
n_num_vals integer;
t_cookies utl_http.cookie_table;
r_cookie utl_http.cookie;
begin
-- Get cookies sent from browser
owa_cookie.get_all(t_names,t_vals,n_num_vals);
-- Set cookies in utl_http
for i in 1..n_num_vals loop
r_cookie.name := t_names(i);
r_cookie.value := t_vals(i);
r_cookie.domain := 'mydomain.com';
r_cookie.expire := sysdate+1;
r_cookie.path := '/';
r_cookie.secure := false;
r_cookie.version := 1;
r_cookie.comment := null;
t_cookies(i) := r_cookie;
end loop;
end forward_cookies;
procedure proc1
is
req utl_http.req;
resp utl_http.resp;
data varchar2(2048);
begin
-- 1. Show cookies available in proc1
htp.p('<br>proc1<br>'
show_cookies;
-- 2. Show cookies available in proc2 when called directly
proc2;
-- 3. Show cookies available in proc2 when called using utl_http
-- Initialize cookies
utl_http.clear_cookies;
forward_cookies;
-- Send request
req := utl_http.begin_request
( 'http://host.mydomain.com:7778/pls/portal/schema.http_test.proc2'
, 'GET'
utl_http.set_header
( req
, 'Accept'
, 'text/plain
, text/html'
utl_http.set_header
( req
, 'User-Agent'
, 'Mozilla/4.0'
-- Fetch and process response
resp := utl_http.get_response( req );
begin
loop
utl_http.read_text(resp, data, 2048);
htp.p(data);
end loop;
exception
when utl_http.end_of_body then null;
end;
utl_http.end_response( resp );
end proc1;
procedure proc2
is
begin
htp.p('<br>proc2<br>'
show_cookies;
end proc2;
end http_test;
After granting this package to portal_public I call proc1 from the browser that returns two equal cookies appropriately for proc1 and proc2 but only portal cookie with a different value that changes with every new request. Sample response below:
proc1
No of cookies: 2
NLS_LANGUAGE = fi-fi%2Bsf%2BFINLAND
portal = 9.0.3+fi-fi+sf+FINLAND+E9DC359A80286AF5E03059C0C47764CA+89C0EE8BC55BAAC80E85148AF3C2000604C0EF4DAEEB15A5F09065ADA0F7D02F1B763785A750735CDC1BF58D1CBED7A6B66FA41DF2B120A014CF9632A2E1AB06CFAB78230700A28D4ED38071B7BAC929E1CC48653949D387
proc2
No of cookies: 2
NLS_LANGUAGE = fi-fi%2Bsf%2BFINLAND
portal = 9.0.3+fi-fi+sf+FINLAND+E9DC359A80286AF5E03059C0C47764CA+89C0EE8BC55BAAC80E85148AF3C2000604C0EF4DAEEB15A5F09065ADA0F7D02F1B763785A750735CDC1BF58D1CBED7A6B66FA41DF2B120A014CF9632A2E1AB06CFAB78230700A28D4ED38071B7BAC929E1CC48653949D387
proc2
No of cookies: 1
portal = 9.0.3+en-us+us+AMERICA+E9DC752B1ED397CAE03059C0C4772C4E+A802192F1D30ABC0790DBB45D30F1FB4E8B876148D14C9AB3BC6D99EF1F72D3E51503FEEDDB9F7ED2C1A7A99C60AFD0ED81945656D3C9AF3C1A926248DA6C00D03DEF6FEB5E869C338B41AAAB0BBD64C1C490DDFD8EE2DE9
Any ideas what's wrong or other ways how Portal page html code could be fetched for manipulation before returning to the browser.