Azure VM persistent page blob disks appear not to deliver 500 IOPs or 60 MB/s as expected

Question

I need to benchmark Azure standard A3 and A4 VMs to see if I can get the advertised
500 IOPs -
60 MB/s per disk.
I'm aware of the 20k maximum IOPS per standard storage account. The following VM test is done in its own (standard) storage account.
I tested a new standard A3 VM that comes with 4 cores, 7 GB RAM, and up to 8x 1TB persistent HDD disks delivering 500 IOPS or 60MB/s each.
I created and used the following script to provision and attach 8x disk to the VM, Powershell into the VM, create a Storage Space - simple, 8 columns, 128k interleave (to make a 1MB stripe - best suited for the anticipated workload):
$SubscriptionName = '####Removed####'
$StorageAccount = 'testdisks1storageaccount'
$VMName = 'testdisks1vm'
$ServiceName = 'testdisks1cloudservice'
$PwdFile = ".\$VMName-####Removed####.txt"
$AdminName = '####Removed####'
# Pick Storage Account
Set-AzureSubscription -SubscriptionName $SubscriptionName -CurrentStorageAccount $StorageAccount
# Create and attach 8x 1TB disks
0..7 | % { $_
Get-AzureVM -ServiceName $ServiceName -Name $VMName |
Add-AzureDataDisk -CreateNew -DiskSizeInGB 1023 -DiskLabel "Disk$_" -LUN $_ |
Update-AzureVM
# Remote to the VM and creacte storage space:
$objVM = Get-AzureVM -Name $VMName -ServiceName $ServiceName
$VMFQDN = (Get-AzureWinRMUri -ServiceName $ServiceName).Host
$Port = (Get-AzureWinRMUri -ServiceName $ServiceName).Port
# Get certificate for Powershell remoting to the Azure VM if not installed already
Write-Verbose "Adding certificate 'CN=$VMFQDN' to 'LocalMachine\Root' certificate store.."
$Thumbprint = (Get-AzureVM -ServiceName $ServiceName -Name $VMName |
select -ExpandProperty VM).DefaultWinRMCertificateThumbprint
$Temp = [IO.Path]::GetTempFileName()
(Get-AzureCertificate -ServiceName $ServiceName -Thumbprint $Thumbprint -ThumbprintAlgorithm sha1).Data | Out-File $Temp
$Cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2 $Temp
$store = New-Object System.Security.Cryptography.X509Certificates.X509Store "Root","LocalMachine"
$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite)
$store.Add($Cert)
$store.Close()
Remove-Item $Temp -Force -Confirm:$false
# Attempt to open Powershell session to Azure VM
Write-Verbose "Opening PS session with computer '$VMName'.."
if (-not (Test-Path -Path $PwdFile)) {
Write-Verbose "Pwd file '$PwdFile' not found, prompting to pwd.."
Read-Host "Enter the pwd for '$AdminName' on '$VMFQDN'" -AsSecureString |
ConvertFrom-SecureString | Out-File $PwdFile
$Pwd = Get-Content $PwdFile | ConvertTo-SecureString
$Cred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $AdminName, $Pwd
$Session = New-PSSession -ComputerName $VMFQDN -Port $Port -UseSSL -Credential $Cred -ErrorAction Stop
# Check available disks:
$ScriptBlock = { Get-PhysicalDisk -CanPool $True }
$Result = Invoke-Command -Session $Session -ScriptBlock $ScriptBlock
$Result | sort friendlyname | FT -a
<#
FriendlyName CanPool OperationalStatus HealthStatus Usage Size PSComputerName
PhysicalDisk2 True OK Healthy Auto-Select 1023 GB testdisks1cloudservice.cloudapp.net
PhysicalDisk3 True OK Healthy Auto-Select 1023 GB testdisks1cloudservice.cloudapp.net
PhysicalDisk4 True OK Healthy Auto-Select 1023 GB testdisks1cloudservice.cloudapp.net
PhysicalDisk5 True OK Healthy Auto-Select 1023 GB testdisks1cloudservice.cloudapp.net
PhysicalDisk6 True OK Healthy Auto-Select 1023 GB testdisks1cloudservice.cloudapp.net
PhysicalDisk7 True OK Healthy Auto-Select 1023 GB testdisks1cloudservice.cloudapp.net
PhysicalDisk8 True OK Healthy Auto-Select 1023 GB testdisks1cloudservice.cloudapp.net
PhysicalDisk9 True OK Healthy Auto-Select 1023 GB testdisks1cloudservice.cloudapp.net
#>
# Create Storage Space with 8 disks, optimize for 1MB stripe
$ScriptBlock = {
$PoolName = "VeeamPool2"
$vDiskName = "VeeamVDisk2"
$VolumeLabel = "VeeamRepo2"
New-StoragePool -FriendlyName $PoolName -StorageSubsystemFriendlyName “Storage Spaces*” -PhysicalDisks (Get-PhysicalDisk -CanPool $True) |
New-VirtualDisk -FriendlyName $vDiskName -UseMaximumSize -ProvisioningType Fixed -ResiliencySettingName Simple -NumberOfColumns 8 -Interleave 128KB |
Initialize-Disk -PassThru -PartitionStyle GPT |
New-Partition -AssignDriveLetter -UseMaximumSize |
Format-Volume -FileSystem NTFS -NewFileSystemLabel $VolumeLabel -AllocationUnitSize 64KB -Confirm:$false
Get-VirtualDisk | select FriendlyName,Interleave,LogicalSectorSize,NumberOfColumns,PhysicalSectorSize,ProvisioningType,ResiliencySettingName,Size,WriteCacheSize
$Result = Invoke-Command -Session $Session -ScriptBlock $ScriptBlock
$Result
<#
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size PSComputerName
F VeeamRepo2 NTFS Fixed Healthy 7.98 TB 7.98 TB testdisks1clouds...
FriendlyName : VeeamVDisk2
Interleave : 131072
LogicalSectorSize : 512
NumberOfColumns : 8
PhysicalSectorSize : 4096
ProvisioningType : Fixed
ResiliencySettingName : Simple
Size : 8778913153024
WriteCacheSize : 0
PSComputerName : testdisks1cloudservice.cloudapp.net
RunspaceId : 36930724-4ed4-4813-a8d5-1a8b56b0b1da
#>
Then I RDP'd into the VM, and ran IOMeter to test the IO performance of the disk I just created. According to Microsoft documentation I'm expecting to see 4K IOPS or 480 MB/s throughput (8x disks x 500 IOPS or 60 MB/s each)
I got:
1.3K IOPS instead of 4k, and ~85MB/s instead of 480MB/s.
I've used all 4 workers (cores) to test, and saturated the disk queue with 20 concurrent IOs (see disk queue length on the bottom right part of the image). I tested with a single IO profile: 64K blocks 50% read/50% write which is the IO profile of the anticipated
workload for this VM. 
This is East US 2 region.
This is consistent with other people's findings like this post.
I also tested using the 8 disks as a striped disk in Windows. I removed the volume, vDisk, Storage Space, then provisioned a traditional RAID 0 striped disk in this Windows Server 2012 R2 VM. Results were slightly better:
This is still far off the expected 4k IOPS or 480 MB/s I should be seeing here.
My question is:
What do I need to do to get this 4k IOPS or 480 MB/s IO performance in standard A3 VM? Or what's the recommended configuration to get this promised performance?
Sam Boutros, Senior Consultant, Software Logic, KOP, PA http://superwidgets.wordpress.com (Please take a moment to Vote as Helpful and/or Mark as Answer, where applicable) _________________________________________________________________________________
Powershell: Learn it before it's an emergency http://technet.microsoft.com/en-us/scriptcenter/powershell.aspx http://technet.microsoft.com/en-us/scriptcenter/dd793612.aspx

Answer 1

OK. BTW, I upgraded this US East 2 region VM to A4, deleted the storage space and retested with a 16-disk, 16-column space. I got exactly twice the performance (2.6k IOPS, 175 MB/s).
See this post for more details.
I repeated the test on another A4 VM in US East region. This was even worse. Showing 1.7k IOPS, 100 MB/s in 16-disk / 16-column storage space disk testing..
Sam Boutros, Senior Consultant, Software Logic, KOP, PA http://superwidgets.wordpress.com (Please take a moment to Vote as Helpful and/or Mark as Answer, where applicable) _________________________________________________________________________________
Powershell: Learn it before it's an emergency http://technet.microsoft.com/en-us/scriptcenter/powershell.aspx http://technet.microsoft.com/en-us/scriptcenter/dd793612.aspx