SCCM 2012 SCCMContentLib folder remains empty on DP

Question

I have a few standalone DP's which are not getting the packages distributed to them
I have 74 Secondary Sites, 1 Primary (no CAS), and 12 standalone DP's
All Secondary sites are operating properly and half of the DP's are not getting any packages disributed to them
Within IIS I have noticed the the DataLib, FileLib and PkgLib virtual directories have not been created, I also noticed that the SCCMContentLib folder on the DP remains empty.
I have cross referenced everything I can think of between the functional DP's and the broken DP's and this seems to be the only thing wrong. I have uninstalled and reinstalled everything (including IIS), rebooted and still getting nowhere with this.
The entire content of the smsdpprov.log is here:
[1BCC][Mon 08/11/2014 14:10:43]:Registry key creation successfull
[1BCC][Mon 08/11/2014 14:10:45]:Registry key creation successfull
[1BCC][Mon 08/11/2014 14:10:45]:Registry key creation successfull
[1BCC][Mon 08/11/2014 14:10:45]:Registry key creation successfull
[1BCC][Mon 08/11/2014 14:10:46]:A DP health monitoring task has been deleted successfully
[1BCC][Mon 08/11/2014 14:10:46]:RegQueryValueExW failed for Software\Microsoft\SMS\DP, PxeInstalled
[1BCC][Mon 08/11/2014 14:10:46]:RegReadDWord failed; 0x80070002
[1BCC][Mon 08/11/2014 14:10:46]:PXE provider is not installed.
[1BCC][Mon 08/11/2014 14:10:46]:Registry key creation successfull
[1BCC][Mon 08/11/2014 14:10:46]:Content library folder
\\?\E:\SCCMContentLib is removed successfully
[3D4][Mon 08/11/2014 14:10:53]:Started process E:\SMS_DP$\sms\bin\vcredist_x64.exe /q /l E:\SMS_DP$\sms\bin\vcredist.log
[3D4][Mon 08/11/2014 14:10:53]:Waiting for the completion of: E:\SMS_DP$\sms\bin\vcredist_x64.exe /q /l E:\SMS_DP$\sms\bin\vcredist.log
[3D4][Mon 08/11/2014 14:10:58]:Failed to run: E:\SMS_DP$\sms\bin\vcredist_x64.exe /q /l E:\SMS_DP$\sms\bin\vcredist.log. Program returned 00000652
[3D4][Mon 08/11/2014 14:11:38]:Started process msiexec.exe /quiet /i E:\SMS_DP$\sms\bin\pulldp.msi /log E:\SMS_DP$\sms\bin\pulldp_install.log
distmgr.log from the secondary site shows "Failed to get usable drives for DP" (I have one dedicated drive with 700 GB free for this purpose):
Signature share exists on distribution point path
\\DP_Site\SMSSIG$ SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:47 AM 1312 (0x0520)
Set share security on share \\DP_Site\SMSSIG$ SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:47 AM 1312 (0x0520)
Set share security on share \\DP_Site\WIN7 PACKAGES SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:47 AM 1312 (0x0520)
Set share security on share \\DP_Site\SCCMContentLib$ SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:47 AM 1312 (0x0520)
for ["Display=\\DP_Site\\DP_Site\"]MSWNET:["SMS_SITE=R32"]\\DP_Site\, failed to get a copy of DP SysResUse from SCF SMS_DISTRIBUTION_MANAGER 8/12/2014
8:53:47 AM 1312 (0x0520)
Failed to get usable drives for DP. SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:47 AM 1312 (0x0520)
Error occurred. Performing error cleanup prior to returning. SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:47 AM 1312 (0x0520)
Cancelling network connection to \\DP_Site\ADMIN$. SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:47 AM 1312 (0x0520)
DP thread with array index 0 ended. SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:47 AM 19400 (0x4BC8)
DP thread with thread handle 0000000000001F30 and thread ID 1312 ended. SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:47 AM 19400 (0x4BC8)
Only retrying local DP update for package pkg00192, no need to replicate package definition to child sites or DP info to parent site. SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:47 AM 19400 (0x4BC8)
StoredPkgVersion (1) of package pkg00192. StoredPkgVersion in database is 1. SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:48 AM 19400 (0x4BC8)
SourceVersion (1) of package pkg00192. SourceVersion in database is 1. SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:48 AM 19400 (0x4BC8)
STATMSG: ID=2302 SEV=E LEV=M SOURCE="SMS Server" COMP="SMS_DISTRIBUTION_MANAGER" SYS=DP_Site SITE=DP1 PID=13636 TID=19400 GMTDATE=Tue Aug 12 12:53:48.005 2014 ISTR0="Test DP" ISTR1="pkg00192"
ISTR2="" ISTR3="" ISTR4="" ISTR5="" ISTR6="" ISTR7="" ISTR8="" ISTR9="" NUMATTRS=1 AID0=400 AVAL0="pkg00192" SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:48 AM 19400
(0x4BC8)
Failed to process package pkg00192 after 34 retries, will retry 66 more times SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:48 AM 19400 (0x4BC8)
Exiting package processing thread. SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:48 AM 19400 (0x4BC8)
Used 0 out of 3 allowed processing threads. SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
The last source update time for pkg pkg00205 is 10/22/2013 6:50:41 PM Eastern Daylight Time SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
Pkg pkg00205 does not have a next start time SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
The last source update time for pkg pkg00505 is 10/22/2013 7:08:07 PM Eastern Daylight Time SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
Pkg pkg00505 does not have a next start time SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
The last source update time for pkg MIL005BD is 10/22/2013 7:13:00 PM Eastern Daylight Time SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
Pkg pkg005BD does not have a next start time SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
The last source update time for pkg pkg00807 is 10/22/2013 7:25:44 PM Eastern Daylight Time SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
Pkg pkg00807 does not have a next start time SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
The last source update time for pkg pkg0017D is 8/11/2014 9:47:34 AM Eastern Daylight Time SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
The next start time for pkg pkg0017D is 8/12/2014 9:47:00 AM Eastern Daylight Time SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
Sleep 1555 seconds... SMS_DISTRIBUTION_MANAGER 8/12/2014 8:53:53 AM 8664 (0x21D8)
Sleep 30 minutes... SMS_DISTRIBUTION_MANAGER 8/12/2014 9:04:46 AM 10176 (0x27C0)

Answer 1

Contents of pulldp_install.log are too big to post... here are the only other errors
[14:12:19] Deleted setup in progress time-stamp.
[14:12:19] Verifying product registration for product {948643F6-8526-4555-93DA-6EBF50CA7650} in MSI database
[14:12:19] Successfully verified product registration for product {948643F6-8526-4555-93DA-6EBF50CA7650} in MSI database
[14:12:19] Registering Product '{948643F6-8526-4555-93DA-6EBF50CA7650}' in WMI
[14:12:19] Successfully registered product '{948643F6-8526-4555-93DA-6EBF50CA7650}' in WMI
[14:12:21] WARNING: Failed to delete setup in progress time-stamp with error 0x80070002
[14:12:21] Starting ccmexec
[14:12:21] Operation complete
[14:12:21] No rollback file detected
Action ended 14:12:21: InstallFinalize. Return value 1.
Action start 14:12:21: RemoveExistingProducts.
Action ended 14:12:21: RemoveExistingProducts. Return value 0.
Action ended 14:12:21: INSTALL. Return value 1.
Property(S): UpgradeCode = {31CEACF7-394A-4FF4-B591-2E7B0E0F90B8}
Property(S): SourceDir = E:\SMS_DP$\sms\bin\
Property(S): TARGETDIR = C:\Windows\CCM\
Property(S): ALLUSERS = 1
Property(S): ErrorDialog = ErrorDialog
Property(S): ProductName = ConfigMgr Distribution Point
Property(S): ProductCode = {948643F6-8526-4555-93DA-6EBF50CA7650}
Property(S): PackageCode = {80D845B1-5F5F-42F0-896C-75A77ABA0AB2}
Property(S): Manufacturer = Microsoft Corporation
Property(S): ProductVersion = 5.00.7804.1000
Property(S): ProductLanguage = 1033
Property(S): REINSTALL = ALL
C:\Windows\CCM\pulldpcmgr.dll
C:\Windows\CCM\PullDP.mof
C:\Windows\CCM\1033\ccmres.dll
Property(S): CcmRegisterPerfCounters = C:\Windows\CCM\
APP:CcmFramework,C:\Windows\CCM\FrameworkPerf.dll,1033|c:\windows\ccm\1033\ccmres.dll
OBJ:CcmEndpoint,160,161,novice,1
CTR:MessagesSucceeded,162,163,65792,novice,0
CTR:MessagesFailed,164,165,65792,novice,0
CTR:MessagesProcessed,208,209,65792,novice,0
CTR:MessagesProcessedPerSecond,210,211,272696320,novice,0
OBJ:CcmMessageQueue,166,167,novice,1
CTR:BytesQueued,168,169,65792,novice,0
CTR:MessagesQueued,170,171,65792,novice,0
CTR:MessagesReceived,200,201,65792,novice,0
CTR:MessagesReceivedPerSec,202,203,272696320,novice,0
CTR:MessagesCompleted,204,205,65792,novice,0
CTR:MessagesCompletedPerSec,206,207,272696320,novice,0
Property(S): CcmUnregisterPerfCounters = C:\Windows\CCM\
APP:CcmFramework,C:\Windows\CCM\FrameworkPerf.dll,1033|c:\windows\ccm\1033\ccmres.dll
Property(S): CcmRegisterPerfCountersRollback = C:\Windows\CCM\
APP:CcmFramework,C:\Windows\CCM\FrameworkPerf.dll,1033|c:\windows\ccm\1033\ccmres.dll
OBJ:CcmEndpoint,160,161,novice,1
CTR:MessagesSucceeded,162,163,65792,novice,0
CTR:MessagesFailed,164,165,65792,novice,0
CTR:MessagesProcessed,208,209,65792,novice,0
CTR:MessagesProcessedPerSecond,210,211,272696320,novice,0
OBJ:CcmMessageQueue,166,167,novice,1
CTR:BytesQueued,168,169,65792,novice,0
CTR:MessagesQueued,170,171,65792,novice,0
CTR:MessagesReceived,200,201,65792,novice,0
CTR:MessagesReceivedPerSec,202,203,272696320,novice,0
Property(S): CcmDetectFilesInUseRemoveRollback_ActionText = Calls CcmDetectFilesInUseRollback. Rolls back the files moved by CcmDetectFilesInUse.
Property(S): CcmDetectFilesInUseRemoveCommit_ActionText = Calls CcmDetectFilesInUseCommit. Commits action of CcmDetectFileInUse. After this the action cannot be rolled back.
Property(S): CcmLookupAccountNames_ActionText = Looks up the name of the Administrators group, Users group, and Creator Owner account, then sets the properties CcmAdministratorGroupName, CcmUsersGroupName, and CcmCreatorOwnerAccountName.
Property(S): CcmRegisterPerfCountersInit_ActionText = Processes the CcmPerfApplication table for components that will be registering a performance counter.
Property(S): CcmRegisterPerfCounters_ActionText = Registers performance counters gathered in the CcmRegisterPerfCountersInit action
Property(S): CcmUnregisterPerfCountersInit_ActionText = Processes the CcmPerfApplication table for components that will be unregistering a performance counter.
Property(S): CcmUnregisterPerfCounters_ActionText = Removes performance counters gathered in the CcmUnregisterPerfCountersInit action
Property(S): CcmRegisterPerfCountersRollback_ActionText = Rolls back any changes made by CcmRegisterPerfCounters if install fails.
Property(S): CcmUnloadWmiProvidersInit_ActionText = Enumerates component table and builds a list of files that will be installed
Property(S): CcmUnloadWmiProviders_ActionText = Enumerates all providers and unloads them, verifies they are unloaded, and then reloads them.
Property(S): CcmTypelibRollbackInit_ActionText = Queues a list of type libraries to be registered on rollback.
Property(S): CcmTypelibRollback_ActionText = In the event of install failing, this event rolls back the type libraries to the state before install started.
Property(S): CcmSetObjectSecurityInit_ActionText = Queries the CcmObjectSecurity table and passes this information to CcmSetObjectSecurity.
Property(S): CcmSetObjectSecurity_ActionText = Applying security permissions
Property(S): CcmSetInstallDir32_ActionText = Property action for setting installation directory for Configuration Manager Client on 32-bit platform.
Property(S): CcmSetInstallDir64_ActionText = Property action for setting installation directory for Configuration Manager Client on 64-bit platform.
Property(S): CcmSetPrimaryFolder_ActionText = Property action for setting target installation folder.
Property(S): PRIMARYFOLDER = TARGETDIR
Property(S): REBOOT = ReallySuppress
Property(S): CcmRegisterHostedApplicationsRollback_ActionText = Rollback the registration of hosted out-of-proc DCOM applications running inside of Configuration Manager Client.
Property(S): CcmRemoveHostedApplicationsInit_ActionText = Prepare to remove the registration of hosted out-of-proc DCOM applications running inside of Configuration Manager Client.
Property(S): CcmRemoveHostedApplications_ActionText = Remove the registration of hosted out-of-proc DCOM applications running inside of Configuration Manager Client.
Property(S): CcmRegisterEndpointsInit_ActionText = Gathers information such as log file name on each endpoint to be registered on the system.
Property(S): CcmRegisterEndpoint_ActionText = Registers endpoint gathered in the action CcmRegisterEndpointsInit with WMI.
Property(S): CcmRegisterEndpointRollback_ActionText = In the event of a failed installation, this action rolls back the changes from CcmRegisterEndpoint.
Property(S): CcmRemoveEndpointsInit_ActionText = Gathers information for each endpoint to be removed from the system.
Property(S): CcmRemoveEndpoint_ActionText = Removes endpoint gathered in the action CcmRemoveEndpointsInit with WMI.
Property(S): CcmRegisterSystemTasksInit_ActionText = Processes the CcmSystemTask table and passes this information to CcmRegisterSystemTask.
Property(S): CcmRegisterSystemTask_ActionText = Registers a System Task with WMI.
Property(S): CcmRegisterSystemTaskRollback_ActionText = Rolls back the changes made by CcmRegisterSystemTask.
Property(S): CcmRemoveSystemTasksInit_ActionText = Processes the CcmSystemTask table and passes this information to CcmRemoveSystemTask.
Property(S): CcmRemoveSystemTask_ActionText = Deletes a System Task from WMI.
Property(S): CcmRegisterProduct_ActionText = If product is registered in MSI database, then product is registered with WMI.
Property(S): CcmRegisterProductRollback_ActionText = Calls CcmUnregisterProduct. Deletes the instance from the WMI in the Configuration Manager namespace using the ProductCode instance in the session.
Property(S): CcmUnregisterProduct_ActionText = Deletes the instance from WMI in ccm namespace using the ProductCode instance in the session.
Property(S): CcmRegisterComponentsInit_ActionText = Generates the list of components for CcmRegisterComponets action to register.
Property(S): CcmRegisterComponents_ActionText = Registers components passed in from CcmRegisterComponentsInit
Property(S): CcmRegisterComponentsRollback_ActionText = In the event of a failed installation, this action rolls back the changes from CcmRegisterComponents.
Property(S): CcmUnregisterComponentsInit_ActionText = Generates the list of components for CcmUnregisterComponets action to unregister.
Property(S): CcmUnregisterComponents_ActionText = Unregisters components passed in from the action CcmUnregisterComponentsInit.
Property(S): CcmSetupLoggingInit_ActionText = Gathers log file information from the LogFile table and passes this information to CcmSetupLogging.
Property(S): CcmSetupLogging_ActionText = Sets component logging information in WMI.
Property(S): SmsShellNotify_ActionText = Notifies the shell for the file-association icon changes.
Property(S): WDS_ActionData = WDS
Property(S): WDS_UnregisterActionData = WDS_UNINST
Property(S): SmsPULLDPDetectDowngrade_ErrorMessage = A newer version of the SCCM Pull Distribution Point is already installed
Property(S): ARPSYSTEMCOMPONENT = 1
Property(S): ARPNOREMOVE = 1
Property(S): ARPNOMODIFY = 1
Property(S): UserExitDialog_Title = Setup Aborted
Property(S): UserExitDialog_SubTitle = Setup was cancelled
Property(S): UserExitDialog_Info = The SCCM Pull Distribution Point setup was cancelled.
Property(S): InstallErrorDialog_Title = Setup Aborted
Property(S): InstallErrorDialog_SubTitle = Setup failed
Property(S): InstallErrorDialog_Info = Setup encountered an error and could not continue.
Property(S): InstallDir.25D3C2C6_30DD_48E8_94A0_93094060B552 = C:\Windows\CCM\
Property(S): CcmDefaultPolicyAuthority = SMS:PULLDP:Default:{948643F6-8526-4555-93DA-6EBF50CA7650}
Property(S): Res1033.0EACD2EE_412B_4985_9727_96BC217AABD8 = C:\Windows\CCM\1033\
Property(S): MsiLogFileLocation = E:\SMS_DP$\sms\bin\pulldp_install.log
Property(S): CCMHTTPSPORT = 443
MSI (s) (1C:4C) [14:12:21:320]: Product: ConfigMgr Distribution Point -- Configuration completed successfully.
MSI (s) (1C:4C) [14:12:21:320]: Windows Installer reconfigured the product. Product Name: ConfigMgr Distribution Point. Product Version: 5.00.7804.1000. Product Language: 1033. Manufacturer: Microsoft Corporation. Reconfiguration success or error status:
0.
=== Logging stopped: 8/11/2014  14:12:21 ===