Intermittent Routing between Shared IP Zones

Question

I've setup a single machine with zones for apache and mail services which use the global zone's external data link. I've setup the zones as shared-ip zones:
zonename: apache
net:
     address: 192.168.0.1/24
     physical: bge1
     defrouter not specified
zonename: mail
net:
     address: 192.168.0.2/24
     physical: bge1
     defrouter not specified
The zones have their routing setup in the global some as such:
route add public apache -interface
route add public mail -interface
And the global ifconfig is as such:
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
     inet 127.0.0.1 netmask ff000000
lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
     zone mail
     inet 127.0.0.1 netmask ff000000
lo0:2: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
     zone apache
     inet 127.0.0.1 netmask ff000000
bge1: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu 1500 index 2
     inet XXX.XXX.XXX.XXX netmask fffffff8 broadcast XXX.XXX.XXX.XXX
     ether 0:23:8b:aa:15:6b
bge1:1: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu 1500 index 2
     zone mail
     inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255
bge1:2: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu 1500 index 2
     zone apache
     inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
The global zone is configured with NAT to map and rdr between the global ip address and the zone's local ip address.
The configuration works and runs ok, but i keep getting connection timeouts about 50% of the time.
I've snooped the tcp connections from the global zone but they are going unanswered even though the zones are running and responding correctly. The ipmon log shows the same behaviour with in requests but no mapped out responses during the connection timeouts.
I think this might be a problem with routing between zones with shared-ip but i'm not sure what i can do to fix the problem?
I'm running Solaris 10 10/09.
Thanks,
Cam

Answer 1

sowmini wrote:
The zones have their routing setup in the global some as such:
route add public apache -interface
route add public mail -interfaceit's not clear what "apache" and "mail" are, in your example above: are these the IP addresses assigned to
each of the non-global zones? (I'm assuming "public" is a subnet that you want the NGZ's to reach?)
yes, apache and mail are the local hostname of the 2 zones which are running those services which is specified in /etc/hosts
apache is 192.168.0.1/24
mail is 192.168.0.2/24
public is the subnet of the global zone's only ip address and external network
>
The global zone is configured with NAT to map and rdr between the global ip address and the zone's local ip address.
The configuration works and runs ok, but i keep getting connection timeouts about 50% of the time.What does "netstat -s -P ip" show? that may tell you where the packets are sporadically getting dropped
Here's the output of running the command:
bash-3.00# netstat -s -P ip
IPv4 ipForwarding = 1 ipDefaultTTL = 255
ipInReceives =8454948 ipInHdrErrors = 0
ipInAddrErrors = 0 ipInCksumErrs = 0
ipForwDatagrams = 152 ipForwProhibits = 0
ipInUnknownProtos = 114 ipInDiscards = 3
ipInDelivers =64396846 ipOutRequests =6476680
ipOutDiscards = 0 ipOutNoRoutes = 238
ipReasmTimeout = 60 ipReasmReqds = 0
ipReasmOKs = 0 ipReasmFails = 0
ipReasmDuplicates = 0 ipReasmPartDups = 0
ipFragOKs = 0 ipFragFails = 0
ipFragCreates = 0 ipRoutingDiscards = 0
tcpInErrs = 3 udpNoPorts = 2435
udpInCksumErrs = 0 udpInOverflows = 0
rawipInOverflows = 0 ipsecInSucceeded = 0
ipsecInFailed = 0 ipInIPv6 = 0
ipOutIPv6 = 0 ipOutSwitchIPv6 = 0
i found this discussion on the networking forum which sounds very similar to what i'm seeing but i've tried to set a static arp for the public router but it doesn't seem to have made much difference:
Solaris Server timeouts
when all is working the media table looks like this:
bash-3.00# netstat -pn
Net to Media Table: IPv4
Device IP Address Mask Flags Phys Addr
bge1 XXX.XXX.XXX.137 255.255.255.255 o 00:0c:31:ec:1b:01
bge1 192.168.0.1 255.255.255.255 SPLA 00:23:8b:aa:15:6b
bge1 192.168.0.2 255.255.255.255 SPLA 00:23:8b:aa:15:6b
bge1 XXX.XXX.XXX.138 255.255.255.255 SPLA 00:23:8b:aa:15:6b
bge1 224.0.0.0 240.0.0.0 SM 01:00:5e:00:00:00
and then every half hour to an hour, the router gets dropped and the table is flushed out before getting re-created:
Net to Media Table: IPv4
Device IP Address Mask Flags Phys Addr
bge1 192.168.0.1 255.255.255.255 SPLA 00:23:8b:aa:15:6b
bge1 192.168.0.2 255.255.255.255 SPLA 00:23:8b:aa:15:6b
bge1 XXX.XXX.XXX.138 255.255.255.255 SPLA 00:23:8b:aa:15:6b
bge1 224.0.0.0 240.0.0.0 SM 01:00:5e:00:00:00