Hyper-V over SMB 3.0 poor performance on 1GB NIC's without RDMA

Question

This is a bit of a repost as the last time I tried to troubleshoot this my question got hijacked by people spamming alternative solutions (starwind) 
For my own reasons I am currently evaluating Hyper-V over SMB with a view to designing our new production cluster based on this technology.  Given our budget and resources a SoFS makes perfect sense.
The problem I have is that in all my testing, as soon as I host a VM's files on a SMB 3.0 server (SoFS or standalone) I am not getting the performance I should over the network.  
My testing so far:
4 different decent spec machines with 4-8gb ram, dual/quad core cpu's, 
Test machines are mostly Server 2012 R2 with one Windows 8.1 hyper-v host thrown in for extra measure.
Storage is a variety of HD and SSD and are easily capable of handling >100MB/s of traffic and 5k+ IOPS
Have tested storage configurations as standalone, storage spaces (mirrored, spanned and with tiering)
All storage is performing as expected in each configuration.
Multiple 1GB NIC's from broadcom, intel and atheros.  The broadcoms are server grade dual port adapters.
Switching has been a combination of HP E5400zl, HP 2810 and even direct connect with crossover cables.
Have tried stand alone NIC's, teamed NIC's and even storage through hyper-v extensible switch.
File copies between machines will easily max out 1GB in any direction.
VM's hosted locally show internal benchmark performance in line with roughly 90% of underlying storage performance.
Tested with dynamic and fixed vhdx's
NIC's have been used in combinations of RSS and TCP offload enabled/disabled.
Whenever I host VM files on a different server from where it is running, I observe the following:
Write speeds within the VM to any attached vhd's are severely effected and run at around 30-50% of 1GB
Read Speeds are not as badly effected but just about manager to hit 70% of 1GB
Random IOPS are not noticeably affected.
Running multiple tests at the same time over the same 1GB links results in the same total through put.
The same results are observed no matter which machine hosts the vm or the vhdx files. 
Any host involved in a test will show a healthy amount of cpu time allocated to hardware interupts.  On a 6 core 3.8Ghz cpu this is around 5% of total.  On the slowest machine (dual core 2.4Ghz) this is roughly 30% of cpu load.
Things I have yet to test:
Gen 1 VM's
VM's running anything other than server 2012 r2
Running the tests on actual server hardware. (hard as most of ours are in production use)
Is there a default QoS or IOPS limit when SMB detects hyper-v traffic?  I just can't wrap my head around how all the tests are seeing an identical bottleneck as soon as the storage traffic goes over smb.
What else should I be looking for? There must be something obvious that I am overlooking!

Answer 1

By nature of a SOFS reads are really good, but there is no write cache, SOFS only seems to perform well with Disk mirroring, this improves the write performance and redundancy but halves your disk capacity.
Mirror (RAID1 or RAID10) actually REDUCES number of IOPS. With read every spindle takes part in I/O request processing (assumimg I/O is big enough to cover the stripe) so you multiply IOPS and MBps on amount of spindles you have in a RAID group and all writes
need to go to the duplicated locations that's why READS are fast and WRITES are slow (1/2 of the read performance). This is absolutely basic thing and SoFS layered on top can do nothing to change this.
StarWind iSCSI SAN & NAS
Not wanting to put the cat amongst the pigeons, this isn't strictly true, RAID 1 and 10 give you the best IOP performance of any Raid group, this is why all the best performing SQL Cluster use RAID 10 for most of their storage requirements,
Features
RAID 0
RAID 1
RAID 1E
RAID 5
RAID 5EE
Minimum # Drives
2
2
3
3
4
Data Protection
No Protection
Single-drive
failure
Single-drive
failure
Single-drive
failure
Single-drive
failure
Read Performance
High
High
High
High
High
Write Performance
High
Medium
Medium
Low
Low
Read Performance (degraded)
N/A
Medium
High
Low
Low
Write Performance (degraded)
N/A
High
High
Low
Low
Capacity Utilization
100%
50%
50%
67% - 94%
50% - 88%
Typical Applications
High End Workstations, data
logging, real-time rendering, very transitory data
Operating System, transaction
databases
Operating system, transaction
databases
Data warehousing, web serving,
archiving
Data warehousing, web serving,
archiving