Starting Over - Making a functional OS X Server

Question

Friends, I need some help here.
I built my OS X Server first as an Advanced Server and frankly it was about as unintuitive as I've ever seen, Novell OES, SUSE Linux, Microsoft were all easier to get the piece parts working. I searched online for books and all of them are either out of date or have extremely poor ratings. I downloaded all the guides from Apple and while nice they are very basic and lack any kind of really useful detail.
I have a mostly working server now or more correctly again. The web server functions properly and I can again see the groups and users pages. Webmail is broken again as is iCal. It runs on the server or is at least running, but I can no longer connect to it from any client workstations. Wikis and blogs are working, I can connect to the server from the Macs and presume that the server is advertising properly on SMB (no Windows workstations here at this time).
The frustration is that I cannot send email from the server (it times out constantly, that I cannot depend upon the configuration (earlier today web services wanted only to connect to the server.domain.com addressing. Webmail had been working and now is not for reasons unknown.
I had hoped to bring the domain in house instead of hosting externally for a number of reasons. Thus far I cannot see doing this due to the ongoing instability of services. I wanted to use OS X server to leverage the incredible success of the Mac workstations, but on a server instead.
I am prepared to blow the whole thing away and start over again to make a fully functional server. I'd appreciate to hear from anyone who has made this work fully and would appreciate guidance in making this so. Until today outbound mail was my only real concern, but I have not yet turned on podcast or quicktime services. I do not use DNS or DHCP as services since I have a router doing DHCP and am passing DNS info through it. The server IP address is statically assigned as one would expect. Port forwarding has been configured on the Airport Extreme for SSH (works) POP and Secure POP (untested - don't like POP), IMAP (works), web 80 (works), web 443 (untested - had issues trying to make https work on the server). I also set up forwarding for port 587 but don't know if it's even necessary. I admit to not being any kind of expert when it comes to Mail servers
I find that although Server Admin and Server Preferences are simple front ends, they lack the richness to get deep (or at least that's my perception so far). I also find that changes to one component has an impact on others (working with mail breaks iCal) and this troubles me greatly.
Thanks to any and all in advance.

Answer 1

Now I'm getting somewhere, or at least I think I am.
Your server should have a static ip outside the dhcp range. assign one first.
Check. Been that way all along.
then, configure the dns service on the server itself via server admin.
create a primary zone for your LAN. it can be real (example.com) or an imaginary, non-routable domain (internal.office). i usually stick with a real domain so that clients can access services inside or out via name instead of monkeying around.
Check outridertechnology.com is the primary zone
create at least one A record for the server itself. when doing this, server admin will automatically create a PTR record for it. forward and reverse name resolution is critical.
Check it created a machine record with the server name leopard1 and it's address 192.168.1.51
with at least that done, enable dns (turn it on).
Check done
verify it's working by adding the server's IP as first resolver in the network preferences (system preferences -> network -> active interface -> dns). i usually add the loopback (127.0.0.1) address and the search domain (the domain for which you created a dns zone above, like example.com).
Check did this. Note now that Safari running on the server can find www.outridertechnology.com when I make the first resolver and search domains 127.0.0.1 and outridertechnology.com but it reports that blogs and wikis are not active. If I go to the web server from a workstation that still only uses the ISP DNS server, I see the wikis and blogs as active. There's probably a step missing by me somewhere.
then dig to verify the forward and reverse lookup. assuming your server's (server.example.com) at 10.12.14.2:
dig server.example.com (should result in an answer pointing to 10.12.14.2)
dig -x 10.12.14.2 (should result in an answer pointing to server.example.com)
Check from Terminal on the server itself dig leopard1.outridertechnology.com points to 192.168.1.51 dig -x on 192.168.1.51 points to leopard1.outridertechnology.com
you can also use host, nslookup… whatever you're used to using.
Umm not sure here. Host reports that outridertechnology.com 's mail is handled by leopard1.outridertechnology.com NSLOOKUP says outridertechnology.com cannot be found, says 127.0.0.1
then, once that's established, i'd add it as the first resolver for the dhcp client range so that all clients are on the same page. if dhcp is running on the airport, add it there within the dhcp range. add the ISP's dns servers as secondary, tertiary. there's also a section within the dns config area of server admin to assign forwarders. add the ISP's dns server's there. in 10.4 and below, this involved a manual edit to the named config file. of course, you can still edit named.conf and the zone files manually, but server admin may step on your changes later if you add features not available via the gui.
Have not changed the router or the clients since it appears that something still is not right
do this and then proceed. i understand you have dns configured externally, but you really need a dns server on the LAN for all this to work.