Latency on saturation, fios tier1 / tier2, abysmal

Question

In Tier4 on [full] link saturation connections retain low latency
In Tier1, Tier2, and some areas Tier3 [full] saturation results in abysmal latency (+300ms).   This is likely a design issue (think ISDN design).
If new service is installed as Tier4 and one downgrades a tier or two the latency issue appears   
* all things being equal *
This is not a CPE issue.  This exists for data from ONT over moca or ethernet. wired ethernet, zero wifi, tcp.  I have seen this on multiple dry loop installs (Tier4 install, downgrade to Tier4)

Answer 1

This is probably a result of Software limiting how fast your speed can go. I see it all the time in setting up QoS for networks. Since the ONTs are receiving a whole 622Mbps downstream if on BPON or a whole 2.4Gbps on GPON, in order to make lines work at a specific speed, Traffic Shaping is required. In this case, that would be the ONT/OLT's job, meaning it is done on a software basis. If it were a hardware basis such as the physical link speed of a connection, your latency should not jump any more than 10ms. If it jumps even more than that, it means that due to the physical link it cannot go any faster and data is either queued or stalled, or there is a huge problem that needs to get fixed. Try it out on an Ethernet connection just for fun. Max out a 100Mbps port in either or both directions, it will go from <1ms latency to 4-9ms latency. Now, take the same connection and limit it using software. You'll see it hold <1ms latency just until you get to the point where the traffic starts to be queued; that is when your latency shoots into the hundreds as a result of shaping.
The same happens on Cable and DSL networks. You can see what I'm talking about on a PowerBoosted cable modem believe it or not. PowerBoost kicks in, your connection goes up 10ms while you're maxing out the channel your modem is on. PowerBoost shuts off, you're at 200+ms latency.
As far as ISDN is concerned, you're going to see latency like that even on such a line, despite it being digital. The technology isn't as mature to handle data as newer methods of data transfers are, especially when it comes down to speed which is what would be choking an ISDN line on latency, not Traffic Shaping (but it does cause the Traffic Shaping effect as a result, just like dial-up suffers from due to speed). It's a physica link issue because of how little bandwidth there is when you're sending or receiving data; 20 byte packets will get queued.
========
The first to bring me 1Gbps Fiber for $30/m wins!