Failover routing design help Needed

Question

Hello.
We are looking to have a setup like this:
                                                      User PCs
                                                            |
                                                            |
                                                            |
                                                          3750x
                                                  (stack - ip base)
                     Servers---------------3750x ---------------- 3750x---------------- Servers
                                            (stack -              (stack -
                                          ip services)         ip services)
                                                 |                         |
                                                 |                         |
                                                 |                         |
                                                 |                         |
                                            Router                 Router
                                                 |                         |
                                                 |                         |
                                                 |                         |
                                              ISP1                   ISP2
We would like to have routing (and vLans) done on the switches, and have internet failover from ISP1 to ISP2 if ISP1 fails, and go back to ISP1 when it comes back up. Trunks between all switches. We also would like to have all devices on the same vLAN if possible.
What is the best approach to do this?
(Note that left and right sides [in brown and green font] are in separate site locations, and that user end [in red font] switches only have ip base, which limits eigrp functionality.)
We tried following this, but doesn't fit our site exactly:
http://www.geekmungus.co.uk/cisco-and-networking/failoverinternetconnectionusingipslatrackingandeigrproutingforinter-sitelinks
(Also ran into issue where switch in the middle would have two routes to internet - so possible issue with priority routes)
Thanks in advance

Answer 1

The servers in SiteB (3750_2) are just replicated copies of the servers in SiteA (3750_1). However, there always needs to be an active connection between SiteA and SiteB. Only time that the UserSite(3750_3) needs to access servers in SiteB is when SiteA is completely down.
However, if the servers in SiteA (3750_1) can still be reached through via site SiteB (3750_2) when the direct connection between the UserSite(3750_3) and SiteA (3750_1) is down, that route should also be available (if possible).
The above is good news because it means we can route optimally for both internet and client to server traffic.  STP is blocking one of the links from 3750_3, the one to 3750_2.
A couple of questions from the configurations you posted -
1) on 3750_3 you said in a previous post the clients use the vlan 10 interface IP on 3750_3 as their default gateway. But there are no default routes in the routing table on 3750_3 so how does internet work at the moment ?
In fact there are no default routes on any of the stacks. Is internet not working at the moment because i can't see how it can be.
What is weird is that you have static default routes configured on each 3750 IP Services stack but they are not even showing up in the routing tables on that stack ?
2) the 192.168.50.32/29 and 192.168.50.40/29 networks. They don't seem to be doing anything ? I think they were meant to be for the uplinks from 3750_3 but they haven't been used for this.
3) the IP services stack connections to the routers. Do you know what IP subnet is used for this. It looks like it is vlan 10 again but can you confirm ie. what are the routers LAN interface IP addresses ?
So if you can answer the above we can probably start with reconfiguration. There is going to be quite a lot. You can however do some prepatory work without affecting anything -
1) pick a new subnet for the client vlan and create the scope on your DHCP servers. If you use both DHCP servers then split the scope in half between DHCP servers
2) pick a new vlan number for use for the clients
3) assuming the router's LAN interfaces are in vlan 10 i will need 4 x /30 subnets for all four uplinks ie. 2 from 3750_3 and 2 for each IP Services stack to router connection. You can just give me a class C if you want and i will break it down
4) we only want the new client vlan on 3750_3. If 3750_3 is in VTP client mode then it will not work once we change to L3 uplinks so we need to change the VTP mode to VTP transparent on the 3750_3. Once you have done this we can then create the new client subnet and the L3 vlan interface for the client subnet + ip helper-addresses and this still won't affect your current setup.
All of the above can be done with no downtime although if you change to VTP transparent do this out of hours just in case there is an impact. There shouldn't be but it is worth being safe.
Next are the changes that will require downtime. I appreciate it is three separate sites but because you are changing the uplinks it needs to be done all at once. Basically we need to -
1) on all clients release the current IP address. Then have them shutdown.
2) modify EIGRP config for new client vlan and remove all the other stuff as it is not needed
3) reconfigure all uplinks to be L3 on switch stacks and then check the routing tables to make sure there is a route for vlan 10.
3) allocate all ports for clients into the new client vlan
4) bring clients back up and they should then get new IPs from the DHCP server
5) remove any unnecessary configuration from 3750_3
6) reconfigure 3750_1 and 3750_2 connections to their routers
7) modify EIGRP configuration on both IP Services stacks
8) modify IP SLA configuration on 3750_1. Note there is no point in tracking on 3750_2 because only when ISP1 fails do you want to use ISP2 and if ISP2 fails while ISP1 is down there is nothing to fail back to.
9) modify EIGRP config on the routers
that's a brief outline of what needs doing so you can see it is a fair bit and will need careful planning. As far as the default route goes there are two options and i'm still thinking it all through -
1) have both routers generate a defaut route and apply a delay on the 3750_3 to 3750_2 uplink so 3750_1 is preferred. The advantage of this is that 3750_3 has both routes so if the uplink to 3750_1 it can immediately switch to 3750_2. The delay would also apply to vlan 10 as well so traffic would go to 3750_1 which is what you want.
2) have only 3750_1 generate a default route and only if ISP1 fails does ISP2 then generate a default route. The advantage of this is that you would not need to tweak the delay to get ti right but if ISP1 fails there will be a delay before 3750_2 realises the default route has gone and generates it's own.
So it's a tradeoff. I'll have another look at the configs etc. and decide which is best.
It is a fairly large redesign but at the moment the configurations are quite confusing because there is a lot of extra stuff which isn't doing anything as far as i can see. I am happy to provide configs and explanations as to what you should see and also happy to be around when you implement it ie. e-mail or if you can't e-mail (no internet access ) then on the phone. You would just need to let me know when you needed me and i'll make sure i am available (no charge obviously).
If you could answer the above questions and let me know how you want to proceed then we can take it from there.
Jon