VPN Server broken with Windows after upgrade from Tiger.

Question

Hey there
I use Tiger 10.4 Server on a PowerMac G4.
It has two network interfaces, one public facing with it's own static IP, and the other internal facing.
The VPN service works perfectly, and allows people to connect via L2TP and assignes them an IP on the internal facing subnet, and allows OS X and Windows clients to connect.
However after upgrading to Leopard, only Mac clients can connect, all the Windows clients connect, and although they get an IP and are able to ping destinations, attempts to connect to these destinations (some of which are web apps on port 80, others are file servers running Samba), they just sit waiting for ever.
I've experimented with this problem, and it appears to be a problem with MTU and packet fragmentation, however these settings appear to be the same between Tiger, which worked, and Leopard which does not work.
Does anyone have any experience with the new VPN Server in Leopard, and can offer me any advice on how to fix this problem? I'm currently downgraded to Tiger again until a fix can be found.

Answer 1

I had the same issue, among others, but I finally got everything to work eventually. It seems that if the IP range of the client connecting to VPN is in the same range of the server LAN, there will be connectivity issues, whether it be pcs and/or macs not being able to connect. The following set up got my VPN services working:
1. Get DNS and Open Directory working properly. When I did an upgrade, the Server Admin updated my zone files with a curious extra space, which killed DNS. For example, I had the name server as ns.company.private., but in the files it would say ns. company.private everywhere! I've been reading about various bugs in upgrading DNS, so I think it's best just to start DNS from scratch. But if you are upgrading, the following thread expalins how to go about setting up DNS and Open Directory: http://discussions.apple.com/thread.jspa?messageID=5957209&#5957209
2. Once you have Open directory users and dns working properly, then set up VPN. Give a unique IP range to the internal network (192.168.7.1/24) that other networks will not emulate. If you use 192.168.1.1, you will likely run into issues. You can always test this method out by changing the IP range from a remote location and trying to get in this way instead of changing the server. Also, be aware that if you use Gateway Assistant within NAT, it will automatically give you a 192.168.1.1/24 range, at least that's been my experience, and this always killed VPN for me. I would set up NAT manually to avoid problems.
3. Ensure that the DNS information under the Client Information tab is correct. For my server I have 192.168.9.1 as the nameserver, and company.private as the search domain. Then set up routing tables. Mine are 192.168.0.0:255.255.0.0 private and 0.0.0.0:0.0.0.0 public.
Also, when you restart the server, stop and restart VPN services, as there is some talk about the Tiger bug still being around, where VPN services are messed up upon startup. This all worked for me and a couple others that had similar server set ups. Hopefully this will work for you.