OS X Server 3.0 new setup -- best practices?

Question

Alright, here's what I'm after.
I'm setting up a completely new OS X Server 3.0 environment.  It's on a fairly new (1.5 year old) Mac Mini, plenty of RAM and disk space, etc.  This server will ONLY be used interally.  It will have a private IP address such as 192.168.1.205 which will be outside of my DHCP server's range (192.168.1.10 to .199) to prevent any IP conflicts.
I am using Apple's Thuderbolt-to-Ethernet dongle for the primary network connection.  The built-in NIC will be used strictly for a direct iSCSI connection to a brand new Drobo b800i storage device.
This machine will provide the following services, rougly in order of importance:
1.  A Time Machine backup server for about 50 Macs running Maverics.
1a.  Those networked Macs will authenticate individually to this computer for the Time Machine service
1b.  This Server will get it's directory information from my primary server via LDAP/Open Directory
2.  Caching server for the same network of computers
3.  Serve a NetInstall image which is used to set up new computers when a new employee arrives
4.  Maybe calendaring and contacts service, still considering that as a possibility
Can anyone tell me the recommended "best practices" for setting this up from scratch?  I've done it twice so far and have faced problems each time.  My most frequent problem, once it's set up and running, is with Time Machine Server.  With nearly 100 percent consistency, when I get Time Machine Server set up and running, I can't administer it.  After a few days, I'll try to look at it via the Server app.  About half the time, there'll be the expected green dot by "Time Machine" indicating it is running and other times it won't be there.  Regardless, when I click on Time Machine, I almost always get a blank screen simply saying "Loading."  On rare occasion I'll get this:
Error Reading Settings
Service functionality and administration may be affected.
Click Continue to administer this service.
Code: 0
Either way, sometimes if I wait long enough, I'll be able to see the Time Machine server setup, but not every time.  When I am able to see it, I'll have usability for a few minutes and then it kicks back to "Loading."
I do see this apparently relevant entry in the logs as seen by Console.app (happens every time I see the Loading screen):
servermgrd:  [71811] error in getAndLockContext: flock(servermgr_timemachine) FATAL time out
servermgrd:  [71811] process will force-quit to avoid deadlock
com.apple.launchd: (com.apple.servermgrd[72081]) Exited with code: 1
If I fire up Terminal and run "sudo serveradmin fullstatus timemachine" it'll take as long as a minute or more and finally come back with:
timemachine:command = "getState"
timemachine:state = "RUNNING"
I've tried to do some digging on these issues and have been greeted with almost nothing to go on.  I've seen some rumblings about DNS settings, and here's what that looks like:
sudo changeip -checkhostname
Primary address = 192.168.1.205
Current HostName = Time-Machine-Server.local
The DNS hostname is not available, please repair DNS and re-run this tool.
dirserv:success = "success"
If DNS is a problem, I'm at a loss how to fix it.  I'm not going to have a hostname because this isn't on a public network.
I have similar issues with Caching, NetInstall, etc.
So clearly I'm doing something wrong.  I'm not upgrading, again, this is an entirely clean install.  I'm about ready to blow it away and start fresh again, but before I do, I'd greatly appreciate any insight from others on some "best practices" or an ordered list on the best way to get this thing up and running smoothy and reliably.

Answer 1

Everything in OS X is dependant on proper DNS.  You probably should start there.  It is the first service you should be configuring and it is the most important to keep right.  Don't configure any services until you have DNS straight.  In OS X, DNS really stands for Do Not Skip.
This may be your toughest decision.  Decide what name you want the machine to be.  You have two choices.
1: Buy a valid domain name and use it on your LAN devices.  You may not have a need now for use externally, but in the future when you use VPN, Profile Manager, or Web Services, at least you are prepared.  This method is called split horizon DNS.  Example would be apple.com.  Internally you may name the server tm.apple.com.  Then you may alias to it vpn.apple.com.  Externally, users can access the service via vpn.apple.com but tm.apple.com remains a private address only.
2: Create an invalid private domain name.  This will never route on the web so if you decide to host content for internal/external use, you may run into trouble, especially with services that require SSL certificates.  Examples might be ringsmuth.int or andy.priv.  These type of domains are non-routable and can result in issues of trust when communicating with other servers, but it is possible.
Once you have the name sorted out, you need to configure DNS.  If you are on a network with other servers, just have the DNS admin create an A and PTR record for you.  If this is your only server, then you need to configure and start the DNS service on Mavericks.  The DNS service is the best Apple has ever created.  A ton of power in a compact tool.  For your needs, you likely need to just hit the + button and fill out the New Device record.  Use a fully qualified host name in the first field and the IP address of your server (LAN address).  You did use a fixed IP address and disabled the wireless card, right?
Once you have DNS working, then you can start configuring your other services.  Time Machine should be pretty simple.  A share point will be created automatically for you.  But before you get here, I would encourage starting Open Directory.  Don't do that until DNS is right and you pass the sudo changeip -checkhostname test.
R-
Apple Consultants Network
Apple Professional Services
Author, "Mavericks Server – Foundation Services" :: Exclusively in the iBooks Store