PXE boot fails and reboots after loading PE

Question

I have run into what I think is a unique issue and need some help determining the cause.
We are in the process of replacing and aging DP/PXE point (2003 R2 SP2) with a new server (2008 STD R2 SP1).  What makes my pxe issue unique is that pxe works without issue on the existing 2003 DP/PXE server.  But on the new 2008 server I run into
the following issue.
Environment:  Config Manager 2007 R2, a single primary, multiple DP's and PXE points.
Issue:  When I attempt to PXE boot a system, I am able to load PE, but shortly after the custom background screen is loaded, the system reboots.  I've searched the internet quite a bit and found lots of potential causes including, bad/missing drivers,
certificate issues, rights issues, etc.  None of these seem to be the cause.
My troubleshooting has determined that the client computers are unable to download the variables.dat file.  I just don't know why.
We're using the same boot images on both servers. 
I've tried using multiple computer models and VM's. 
I've opened a command prompt as soon as our background image loads and have verified that the system is pulling a valid IP address.  I am able to map a drive to the PXE server's REMINST share using our sccm net access account and manually copy the .var
file using xcopy to the appropriate directory on the local virtual drive.   I've also attempted to manually run smstftp.exe by mimicking the command line from the smsts log file.  I'll admit that I'm not sure I have the correct syntax for smstftp. 
I've tried several variations and all but one result in a short pause and no file copied/created in the X:\sms\data folder.  The one that does produce a result says that the file cannot be found.  I checked for typos and made sure I used the name
of the newly created .var file.
I've also tried disabling anti-virus on the server, shutting off the windows firewall on the server, granting everyone read rights to the REMINST share.
Below is the smstslog file I've been using as a reference.  Per corporate security policy, I have X'd out the IP address of the PXE server.  The log file for the successful pxe boot from the 2003 server shows an exit code of 0 for smstftp.exe, a note
about successful download of the pxe var file and then it continues through the rest of the boot process.
-----SMSTS log file from a failed PXE boot on the new 2008 server -----
<![LOG[LOGGING: Finalize process ID set to 832]LOG]!><time="16:13:54.440+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="836" file="tslogging.cpp:1489">
<![LOG[==============================[ TSBootShell.exe ]==============================]LOG]!><time="16:13:54.440+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="836"
file="bootshell.cpp:963">
<![LOG[Succeeded loading resource DLL 'X:\sms\bin\i386\1033\TSRES.DLL']LOG]!><time="16:13:54.440+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="836" file="util.cpp:869">
<![LOG[Debug shell is enabled]LOG]!><time="16:13:54.440+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="836" file="bootshell.cpp:974">
<![LOG[Waiting for PNP initialization...]LOG]!><time="16:13:54.471+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="bootshell.cpp:59">
<![LOG[Booted from network (PXE)]LOG]!><time="16:13:54.830+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="configpath.cpp:198">
<![LOG[Found config path X:\sms\data\]LOG]!><time="16:13:54.830+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="bootshell.cpp:553">
<![LOG[Booting from removable media, not restoring bootloaders on hard drive]LOG]!><time="16:13:54.830+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="bootshell.cpp:564">
<![LOG[Executing command line: wpeinit.exe -winpe]LOG]!><time="16:13:54.830+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="bootshell.cpp:767">
<![LOG[Executing command line: X:\WINDOWS\system32\cmd.exe /k]LOG]!><time="16:13:57.014+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="836" file="bootshell.cpp:767">
<![LOG[The command completed successfully.]LOG]!><time="16:13:57.014+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="836" file="bootshell.cpp:850">
<![LOG[Successfully launched command shell.]LOG]!><time="16:13:57.014+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="836" file="bootshell.cpp:430">
<![LOG[The command completed successfully.]LOG]!><time="16:14:41.458+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="bootshell.cpp:850">
<![LOG[Starting DNS client service.]LOG]!><time="16:14:41.458+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="bootshell.cpp:597">
<![LOG[Executing command line: X:\sms\bin\i386\TsmBootstrap.exe /env:WinPE /configpath:X:\sms\data\]LOG]!><time="16:14:41.973+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880"
file="bootshell.cpp:767">
<![LOG[The command completed successfully.]LOG]!><time="16:14:41.973+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="bootshell.cpp:850">
<![LOG[==============================[ TSMBootStrap.exe ]==============================]LOG]!><time="16:14:41.989+000" date="12-23-2013" component="TSMBootstrap" context="" type="1" thread="1932"
file="tsmbootstrap.cpp:1039">
<![LOG[Command line: X:\sms\bin\i386\TsmBootstrap.exe /env:WinPE /configpath:X:\sms\data\]LOG]!><time="16:14:41.989+000" date="12-23-2013" component="TSMBootstrap" context="" type="0" thread="1932"
file="tsmbootstrap.cpp:1040">
<![LOG[Succeeded loading resource DLL 'X:\sms\bin\i386\1033\TSRES.DLL']LOG]!><time="16:14:41.989+000" date="12-23-2013" component="TSMBootstrap" context="" type="1" thread="1932" file="util.cpp:869">
<![LOG[Succeeded loading resource DLL 'X:\sms\bin\i386\TSRESNLC.DLL']LOG]!><time="16:14:41.989+000" date="12-23-2013" component="TSMBootstrap" context="" type="1" thread="1932" file="resourceutils.cpp:152">
<![LOG[Processor Is IA64: 0]LOG]!><time="16:14:41.989+000" date="12-23-2013" component="TSMBootstrap" context="" type="1" thread="1932" file="tsmbootstrap.cpp:1005">
<![LOG[PXE Boot with Root = X:\]LOG]!><time="16:14:41.989+000" date="12-23-2013" component="TSMBootstrap" context="" type="1" thread="1932" file="tsmbootstrap.cpp:921">
<![LOG[Executing from PXE in WinPE]LOG]!><time="16:14:41.989+000" date="12-23-2013" component="TSMBootstrap" context="" type="0" thread="1932" file="tsmbootstrap.cpp:936">
<![LOG[Loading TsPxe.dll from X:\sms\bin\i386\TsPxe.dll]LOG]!><time="16:14:41.989+000" date="12-23-2013" component="TSMBootstrap" context="" type="0" thread="1932" file="tsmbootstraputil.cpp:1319">
<![LOG[TsPxe.dll loaded]LOG]!><time="16:14:42.004+000" date="12-23-2013" component="TSPxe" context="" type="0" thread="1932" file="tsmbootstraputil.cpp:1329">
<![LOG[Device has PXE booted]LOG]!><time="16:14:42.004+000" date="12-23-2013" component="TSPxe" context="" type="0" thread="1932" file="tspxe.cpp:122">
<![LOG[Variable Path: \SMSTemp\2013.12.23.16.11.24.0002.{AB0FBE86-1F6C-47D7-919B-A44641035A2E}.boot.var]LOG]!><time="16:14:42.004+000" date="12-23-2013" component="TSPxe" context="" type="0" thread="1932"
file="tspxe.cpp:134">
<![LOG[Variable Key Len: 61]LOG]!><time="16:14:42.004+000" date="12-23-2013" component="TSPxe" context="" type="0" thread="1932" file="tspxe.cpp:141">
<![LOG[Succesfully added firewall rule for Tftp]LOG]!><time="16:14:42.004+000" date="12-23-2013" component="TSPxe" context="" type="0" thread="1932" file="fwopen.cpp:123">
<![LOG[Executing: X:\sms\bin\i386\smstftp.exe -i XXX.XXX.XXX.XXX get \SMSTemp\2013.12.23.16.11.24.0002.{AB0FBE86-1F6C-47D7-919B-A44641035A2E}.boot.var X:\sms\data\variables.dat]LOG]!><time="16:14:42.004+000" date="12-23-2013"
component="TSPxe" context="" type="0" thread="1932" file="tspxe.cpp:177">
<![LOG[Command line for extension .exe is "%1" %*]LOG]!><time="16:14:42.067+000" date="12-23-2013" component="TSPxe" context="" type="0" thread="1932" file="commandline.cpp:229">
<![LOG[Set command line: "X:\sms\bin\i386\smstftp.exe" -i XXX.XXX.XXX.XXX get \SMSTemp\2013.12.23.16.11.24.0002.{AB0FBE86-1F6C-47D7-919B-A44641035A2E}.boot.var X:\sms\data\variables.dat]LOG]!><time="16:14:42.067+000" date="12-23-2013"
component="TSPxe" context="" type="0" thread="1932" file="commandline.cpp:707">
<![LOG[Executing command line: "X:\sms\bin\i386\smstftp.exe" -i XXX.XXX.XXX.XXX get \SMSTemp\2013.12.23.16.11.24.0002.{AB0FBE86-1F6C-47D7-919B-A44641035A2E}.boot.var X:\sms\data\variables.dat]LOG]!><time="16:14:42.067+000" date="12-23-2013"
component="TSPxe" context="" type="1" thread="1932" file="commandline.cpp:805">
<![LOG[Process completed with exit code 1]LOG]!><time="16:15:29.179+000" date="12-23-2013" component="TSPxe" context="" type="1" thread="1932" file="commandline.cpp:1102">
<![LOG[Succesfully removed firewall rule for Tftp]LOG]!><time="16:15:29.194+000" date="12-23-2013" component="TSPxe" context="" type="0" thread="1932" file="fwopen.cpp:146">
<![LOG[uExitCode == 0, HRESULT=80004005 (e:\nts_sms_fre\sms\server\pxe\tspxe\tspxe.cpp,185)]LOG]!><time="16:15:29.194+000" date="12-23-2013" component="TSPxe" context="" type="0" thread="1932"
file="tspxe.cpp:185">
<![LOG[Failed to download pxe variable file. Code(0x00000001)]LOG]!><time="16:15:29.194+000" date="12-23-2013" component="TSPxe" context="" type="3" thread="1932" file="tspxe.cpp:185">
<![LOG[PxeGetPxeData failed with 0x80004005]LOG]!><time="16:15:29.194+000" date="12-23-2013" component="TSPxe" context="" type="3" thread="1932" file="tsmbootstraputil.cpp:1419">
<![LOG[HRESULT_FROM_WIN32(dwError), HRESULT=80004005 (e:\nts_sms_fre\sms\client\tasksequence\tsmbootstrap\tsmbootstraputil.cpp,1420)]LOG]!><time="16:15:29.194+000" date="12-23-2013" component="TSPxe" context=""
type="0" thread="1932" file="tsmbootstraputil.cpp:1420">
<![LOG[TSMBootstrapUtil::PxeGetPxeData(&bPxeBooted, sVariablesFile, sPxePasswd), HRESULT=80004005 (e:\nts_sms_fre\sms\client\tasksequence\tsmbootstrap\tsmediawizardcontrol.cpp,2236)]LOG]!><time="16:15:29.194+000" date="12-23-2013"
component="TSPxe" context="" type="0" thread="1932" file="tsmediawizardcontrol.cpp:2236">
<![LOG[oTSMediaWizardControl.Run( sMediaRoot, true, true ), HRESULT=80004005 (e:\nts_sms_fre\sms\client\tasksequence\tsmbootstrap\tsmbootstrap.cpp,937)]LOG]!><time="16:15:29.194+000" date="12-23-2013" component="TSPxe"
context="" type="0" thread="1932" file="tsmbootstrap.cpp:937">
<![LOG[Execute( eExecutionEnv, sConfigPath, sTSXMLFile, uBootCount, &uExitCode ), HRESULT=80004005 (e:\nts_sms_fre\sms\client\tasksequence\tsmbootstrap\tsmbootstrap.cpp,1106)]LOG]!><time="16:15:29.194+000" date="12-23-2013"
component="TSPxe" context="" type="0" thread="1932" file="tsmbootstrap.cpp:1106">
<![LOG[Exiting with return code 0x80004005]LOG]!><time="16:15:29.194+000" date="12-23-2013" component="TSPxe" context="" type="1" thread="1932" file="tsmbootstrap.cpp:1118">
<![LOG[Execution complete.]LOG]!><time="16:15:29.194+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="bootshell.cpp:624">
<![LOG[Finalizing logging from process 832]LOG]!><time="16:15:29.194+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="tslogging.cpp:1736">
<![LOG[Finalizing logs to root of first available drive]LOG]!><time="16:15:29.194+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="tslogging.cpp:1578">
<![LOG[LOGGING: Setting log directory to "D:\SMSTSLog".]LOG]!><time="16:15:29.491+000" date="12-23-2013" component="TSBootShell" context="" type="1" thread="880" file="tslogging.cpp:1803">
This has been an extremely frustrating issue and any assistance would be greatly appreciated!

Answer 1

Thanks for your quick response Jason!  I didn't expect someone to reply so quickly or I would have checked back sooner. 
I had found the two 'older' posts already, but had not seen the 'newer' one.  Unfortunately that did not give me any new ideas.  But your comment on checking for TFTP availability did.  Here are things I have tried since my original
post...
I re-ran most of my tests in case I missed something.  I only found one change.  Even though I double-checked, I must have made a typo when I manually ran the smstftp.exe command, because when I ran it again I received a timeout message instead
of file not found.
I had a minor 'thinking outside of the box moment' and decided to PXE boot the new 2008 R2 server itself.  This was successful and I interpreted the success as meaning that the hardware is ok.  Thinking there may be a compatibility issue with
the hardware and the OS, I tried a few different NIC drivers, settings, registry keys, and even a completely different NIC.  No luck on any of these.
I decided to build another Server on a VM tovalidate my build process and configuration.  And of course clients in multiple locations were able to PXE boot off this VM.  Too bad I can't use this in production.
After reading your response Jason, I began to focus on network.  I moved the server to a few different locations so it was utilizing different switches.  No luck.  I noticed in the event viewer for WDS that the server was logging the
beginning of the boot.var file via TFTP.  This of course was not very surprising.  What was surprising is that the very next entry (informational) noted that the client 'COMPLETED' the download of the boot.var file via TFTP.  I know that completed
does not mean successful, but it usually implies or is interpreted as successful.  It should have logged a warning or error, or nothing at all because although the process completed, it was not successful.  I re-verified that the file was not downloaded
to the client and the client log file still shows the same error noted in the logfile from my original post.
Finally, I installed sniffing software on the server and ran some captures while attempting to PXE boot.  Even though I am not much of a network guy, I quickly discovered two things.  First, I found the section where the client attempts to download
the boot.var file.  Unfortunately I don't think the local security team will allow me to post the capture, so I'll do my best to describe what I found.  It starts with a single entry where the client calls for the file via TFTP protocol.  This
is followed by a series of alternating entries (all TFTP) where it looks like the server attempts to send a portion of the file, and the client sends an acknowledgement.  The sending entries all have checksum errors.  The checksum received on
all packets is 0x0000 and of course should be something else.  There is also a shorter section below this with alternating entries where the server attempts to send ICMP packets and the client responds with TFTP acknowledgements. 
The ICMP entries are all marked as Destination unreachable (Port Unreachable).
The second thing I noticed from the network capture is the a few 'Spanning Tree Protocol' entries.  I my search for a solution, I remember reading several posts saying that Spanning Tree can cause this issue.  When I asked, I was assured that
Spanning Tree was disabled in this environment.  It made sense too, because the 2003 PXE server was functioning properly, and Spanning Tree should affect both 2003 and 2008 servers, right?
Either way I will bring my findings to the network team and see what they have to say.
Any additional thoughts or ideas???