Pull DP issues after applying throttling.

Question

After coming across this thread here https://social.technet.microsoft.com/Forums/en-US/97cc527d-9988-4b6c-bf09-743f9c272338/throttling-to-distribution-points?forum=configmanagergeneral&prof=required I'm wondering if BITS policy is causing my pull
dps to fail intermittently or if their failing just because their pull dps. We've got about 110 up and running now. Also as i've been working with the MS help desk for about a week at this point i'm wondering if its a product issue not client specific. 
We've got about 500 locations with just under 300 of those going to be, running as pull dps. We're in the middle of a migration right now and were about half way through. That when I noticed some of the DPs are having problems. I have provided better detail
in the thread i just referenced. It seems like there is an issue either because of bits throttling or because of the nature of pulldps. Could anyone shine further light on what Andy made reference to toward the end of the thread i linked in the beginning because
this is putting me through the ringer!

Answer 1

Hey David thanks for sharing. Your environment sounds very similar to mine in size anyways. Some additional detail on my setup: 1 primary with ALL the main roles(i.e. FSP, MP, DP, WSUS, SUP, SCEP, and SQL is co-located on the server all running under a PKI
infrastructure), 2 more servers also in our datacenter, provisioned as just push dps ATM but built so that if need be SQL could be spun up and off load site roles or expand to include a downstream SUP point. 
A couple of things worth mentioning. I did a brief overview of what the CUs fixed and didn't see anything specific called out that would resolve the problems that were having. I opened a ticket with MS and on a side note those techs are generally somewhat
knowledgeable but their track record with me is rather horrid. I've ended up solving 12 of 15 cases. The most recent ticket that I had open was in regard to the large amount of Pull DPs that are seeming to fail. We were able to get it healthy again by just
doing a bunch of troubleshooting. While it took a couple very long nights it was worth it. I learned quite a bit about how content replicates and why the DPs were failing. 
As I'm sure that your aware, a pull DP is simply a client that pull content from the source push DP and then does the SIS(single instance storage) and dedup processing on the pull as opposed to it being handled by the source push dp. We identified 2 errors
of concern through testing. The cert maint log was complaining about 2 missing registry keys, and content was failing to replicate. We would open the DTS log on the pull and check the bits transfer either with bits admin or posh get-bitstransfer and see it
failing out after sometime. We added the keys it was complaining about as the tech had a KB in the HD system, and while that issue was resolved through this method post SP1 pre R2 that didn't seem to resolve it. In the end i checked a healthy pull DP and it
was complaining about the same missing registry keys. What got me to a healthy pull DP was reinstalling the SCCM client on the pull DP.
Now i just need to do that 30 more times approx. I might still look into deploying the CUs prior to trying to run through all them DPs as that is alot of work but first i have to deal with patch Tuesday duties. Allow me to stand on my soap box for a moment.
I am in the very unfortunate position of not only migrating a very large SCCM 2012 env. but I have to maintain both env. and continue on with the day to day operations. Deploying software, e.g. Java, Flash, Reader, etc. I have to create the patch packages
and test/deploy to the entire enterprise. Finally i have to keep up with the OSD piece as well. Needless to say I've been averaging about 80 hour work weeks and will probably continue to do so for the next few months until this migration is completed. Its
suicide! Every SCCM admin that i've talked with has said this project would fail because we are so under staffed. I've been killing myself to make sure that doesn't happen. So enough whining.
The last thing I want to say is I ran into yet another problem on all days patch Tuesday, two days ago. The WSUS app pool crashed. I upped its memory to 5 gigs from 1.8 and it crashed again. I upped it to 7, same. So then i upped it to just under 9 and it
stayed up. Looking like i'm going to have to up it to if my calculations are correct about 12 or 14 gigs to support our entire environment. I didn't know if the WSUS app pool went down client would come back in mass trying to grab meta data at essentially
the same time. Learned that the hard way. I've read that they will transfer between 5-10 megs each. This month was a brutal month for patches. One of the largest i've ever seen. Wanted to share that tidbit in case someone else happens to run into a similar
problem.
Finally we have a pressing issue that has everyone up to the CIO involved, and i"m about to open a new question on here as i can't find the answer after several days of looking. Clients are going out to the internet for FEP/SCEP udpates if they've been
off the network for more than 7 days and since networking only has QOS on the datacenter subnets its crushing the facilities circuits. We need the exact URL they're trying to go to and I simply can't find it. I looked inside the MPCMDRUN.Log and it just shows
go.microsoft.com/fwlink? and what worse that gets redirected to a load balanced partner of theirs...Akamai from what our Pathview appliances are showing us. If we can't get the exact link then we can't properly throttle it.
My manager wants me to put a BITS throttling client setting in place across the board for client machines, something like 64k, but i think that has the potential to make things worse not better. I say worse because now instead of pulling the 120 meg update
quickly i.e. about 10 to 15 minutes it will take 7 hours. From what i've read bits traffic from many  hosts will stack up and is unaware of other production traffic. So it will increase the chance of saturating the link, not lessening it. I could be wrong.
Thats just what i gathered from one site i was reading.
If you have read this far, how are you dealing with SCEP/FEP cleints pulling their over 7 day updates over WAN circuits? I always thought they would pull that data from the SCCM environment and it appears that they initially try but fail then immediately
resort to going out to the internet. I'm thinking about possibly using GPO for SCEP/FEP even though it could potentially cause a conflict. I'll test it thoroughly if i do try that method. The most crazy thing about this entire diatribe that blew my mind is
every single client that migrates from 2007 to 2012 goes out to Microsoft and pulls 120 megs of data. Then they use the SCCM env. for their def. I had no idea that was happening until it start causing issues for a single site. I've tested in on 5 devices and
sure enough all 5 went to the internet immediately after being migrated.
Well this got way to long. My apologies. Thanks again for your input on the CUs. I will look into deploying them when I get some time, perhaps this weekend...or next. Wanted to go try for my 243 this weekend.