Ipfilter: does policy routing work on Solaris 10?

Question

Hello,
- Does the ipf redirection (aka policy routing) feature work with the
ipfilter that comes with Solaris 10?
I would like to use the the ipf redirection statements "to
interface:router_ip" or "reply-to interface:router_ip" as decribed in
http://coombs.anu.edu.au/~avalon/ipf.new.txt
(The syntax is mentionned in the BNF of the Solaris 10 ipf(4) man
page, but the explanations there are lacking.)
On a machine that has two interfaces, the purpose is to send output
reply packets of a TCP session to the same interface that the input
packets came from. The idea to use ipfilter to do this comes from the
blog entry:
Packets out of the wrong interface
http://blogs.sun.com/carlson/entry/packets_out_of_the_wrong
My first try was to use "reply-to" in a "keep state" rule:
pass in quick on e1000g305000 reply-to e1000g305000:10.13.5.1 proto tcp from any to any port = 443 keep state keep frags group i_sso-test1
Which I understand as "once a connection to port 443 starts on
interface e1000g305000 send all reply packets to the same interface to
the gateway 10.13.5.1"
But it does not work; in the ipf log it shows that the rule matched:
22:56:32.770690 e1000g305000 @i_sso-test1:1 p 10.194.17.11,5648 -> 10.13.5.181,443 PR tcp len 20 60 -S K-S K-F IN
22:56:32.770783 e1000g0 @i_sso-test1:1 p 10.13.5.181,443 -> 10.194.17.11,5648 PR tcp len 20 44 -AS K-S K-F OUT
But the reply packet is not seen on the router (10.13.5.1), nor does
it get to 10.194.17.11 through another route (no firewall on that
machine).
My second try was to use two stateless rules, and to do "source port
routing" for outgoing packets:
pass in quick proto tcp from any to any port = 443 group i_sso-test1
pass out quick on e1000g0 to e1000g305000:10.13.5.1 proto tcp from any port = 443 to any group o_sso-test1
pass out quick proto tcp from any port = 443 to any group o_sso-test1
Which I understand as "incoming packets to port 443 are allowed and
outgoing packets from port 443, if passing on interface e1000g0, are
redirected through interface e1000g305000 via the gateway 10.13.5.1,
if not, are just allowed".
It does not work either; in the ipf log it shows that both the in and
the first out rules matched:
23:09:00.591163 e1000g305000 @i_sso-test1:1 p 10.194.17.11,26080 -> 10.13.5.181,443 PR tcp len 20 60 -S IN
23:09:00.591363 e1000g0 @o_sso-test1:1 p 10.13.5.181,443 -> 10.194.17.11,26080 PR tcp len 20 44 -AS OUT
But again the reply packet seems to be lost in thin air.
I have tried various other rules to no avail.
- Should this work with ipfilter v4.1.9 (592) coming with Solaris 10
u7?
- Am I missing something in the configuration?
- Shouldn't the ipf log show the outgoing reply packet twice? (Once on
the "wrong" interface e1000g0 and once on the interface it is
redirected to e1000g305000.) Or indicate in another manner that the
redirection occurred (like it indicates K-S for "keep state")?
Context:
# netstat -rn
Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
default 10.194.7.1 UG 1 2407
default 10.194.7.1 UG 1 5104 e1000g0
10.13.5.0 10.13.5.181 U 1 5 e1000g305000:1
10.194.7.0 10.194.7.81 U 1 3 e1000g0:2
224.0.0.0 10.194.7.81 U 1 0 e1000g0:2
127.0.0.1 127.0.0.1 UH 1 7 lo0:7
# cat /etc/release
Solaris 10 5/09 s10s_u7wos_08 SPARC
Copyright 2009 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 30 March 2009
# ipf -V
ipf: IP Filter: v4.1.9 (592)
Kernel: IP Filter: v4.1.9
Running: yes
Log Flags: 0x70000000 = pass, block, nomatch
Default: pass all, Logging: available
Active list: 0
Feature mask: 0x107
If it matters, this is occuring in a Solaris 10 zone, whith virtual
interfaces one of which uses 801.q tagging (vlan 305, subnet
10.13.5.0/24), and the "router" is a Cisco ACE load balancer with
interface 10.13.5.1 on the server side.
Thanks in advance for your help in this matter!
Best regards,
Dominique
Mr Dominique Petitpierre Email: User@Domain
Division Informatique User=Dominique.Petitpierre
University of Geneva Domain=unige.ch

Answer 1

I was saying
If it matters, this is occurring in a Solaris 10 zone, whith virtual
interfaces one of which uses 801.q tagging (vlan 305, subnet
10.13.5.0/24),...Well, it turns out that 802.1q tagging does matter: packets redirected
by an ipf policy based routing rule to an interface with tagging are
not transmitted.
In order to better see what was happening the ipf rules were extended
like this (stateless case):
@1 pass in quick on e1000g0 proto tcp from any to any port = 443 group i_sso-test1
@2 pass in quick on e1000g305000 proto tcp from any to any port = 443 group i_sso-test1
@1 pass out quick on e1000g0 to e1000g305000:10.13.5.1 proto tcp from 10.13.5.181/32 port = 443 to any group o_sso-test1
@2 pass out quick on e1000g305000 to e1000g0:10.194.7.1 proto tcp from 10.194.7.81/32 port = 443 to any group o_sso-test1
@3 pass out quick on e1000g305000 proto tcp from any port = 443 to any group o_sso-test1
@4 pass out quick on e1000g0 proto tcp from any port = 443 to any group o_sso-test1Also, for the purpose of the demonstration, the zone configuration was
modified to direct all packets to the same interface with tagging,
thus having just one default route:
zonecfg -z sso-test1 info net
net:
        address: 10.13.5.181/24
        physical: e1000g305000
        defrouter: 10.13.5.1
net:
        address: 10.194.7.81/24
        physical: e1000g305000
        defrouter: 10.13.5.1
netstat -rn
Routing Table: IPv4
  Destination           Gateway           Flags  Ref     Use     Interface
default              10.194.7.1           UG        1       2867          
default              10.13.5.1            UG        1         86 e1000g305000
10.13.5.0            10.13.5.181          U         1          2 e1000g305000:1
10.194.7.0           10.194.7.81          U         1          0 e1000g305000:3
224.0.0.0            10.13.5.181          U         1          0 e1000g305000:1
127.0.0.1            127.0.0.1            UH        1          7 lo0:7     (In this peculiar case the default route to 10.194.7.1 is an artifact
displayed by netstat due to the zone isolation mechanism, but it is
not actually used for routing at the zone level; the interface without
tagging, e1000g0, is only displayed on the global zone where ipfilter
operates)
When testing from 10.194.17.11 with "telnet 10.13.4.180 443", it
works. And one can see in the ipf logs that it is the third out rule
that matched (@o_sso-test1:3), i.e. there was no redirection on
another interface (proof that there is nothing wrong with the context
setup):
16:59:30.479660 e1000g305000 @i_sso-test1:2 p 10.194.17.11,2111 -> 10.13.5.181,443 PR tcp len 20 60 -S IN
16:59:30.479844 e1000g305000 @o_sso-test1:3 p 10.13.5.181,443 -> 10.194.17.11,2111 PR tcp len 20 44 -AS OUT
16:59:30.480182 e1000g305000 @i_sso-test1:2 p 10.194.17.11,2111 -> 10.13.5.181,443 PR tcp len 20 40 -A INWhen testing from 10.194.17.11 with "telnet 10.194.7.81 443", it works
also. This time one can see in the ipf logs that it is the second out
rule that matched (@o_sso-test1:2), i.e. there was redirection from
e1000g305000 to e1000g0.
16:59:41.247101 e1000g0 @i_sso-test1:1 p 10.194.17.11,3851 -> 10.194.7.81,443 PR tcp len 20 60 -S IN
16:59:41.247206 e1000g305000 @o_sso-test1:2 p 10.194.7.81,443 -> 10.194.17.11,3851 PR tcp len 20 64 -AS OUT
16:59:41.247508 e1000g0 @i_sso-test1:1 p 10.194.17.11,3851 -> 10.194.7.81,443 PR tcp len 20 52 -A INA packet capture confirms this and one can see in the capture the
SYN-ACK reply packet go out on e1000g0.
The reverse case, essentially the original setup shown in my first
post, where the default route is the interface without tagging
(e1000g0) and the reply packet matches the redirection rule from
e1000g0 to the interface with tagging e1000g305000, the packet is lost
(i.e. is not visible in the packet capture on either interface).
Further tests with stateful redirection ("reply-to") show the same
pattern (does not work when packets are redirected to an interface
with tagging).
It looks like it is a bug: may be ipfilter injects the redirected
packet at a processing stage where it should already have a 802.1q tag
but does not, or something similar; in the working case, ipfilter acts
on a not yet tagged packet which can be used "as is" at the same
processing stage on the non tagging interface, and thus is correctly
transmitted.
Conclusion: ipfilter policy based routing does work on Solaris 10u7,
but, at least in my setup, not when redirection occurs to a 802.1q
tagging interface.
- Could somebody confirm this?
- Is this a known bug? (I didn't find anything relevant on sunsolve or
on the ipfilter mailing list)
Edited by: kleinstein on Oct 1, 2009 4:22 AM
Edited by: kleinstein on Oct 1, 2009 4:25 AM
Edited by: kleinstein on Oct 1, 2009 4:30 AM
Edited by: kleinstein on Oct 1, 2009 4:32 AM
Edited by: kleinstein on Oct 1, 2009 4:37 AM
Edited by: kleinstein on Oct 1, 2009 4:40 AM
Edited by: kleinstein on Oct 1, 2009 4:41 AM