Disk Transfer (reads and writes) Latency is Too High

Question

i keep getting this error:
the Logical Disk\Avg. Disk sec/Transfer performance counter  has been exceeded.
i got these errors on the following servers:
active directory
SQL01 (i have 2 sql clustered)
CAS03 (4 cas server loadbalanced)
HUB01
MBX02(Clustered)
a little info on our environment:
*Using SAN storage.
*Disks are new ,and working fine
*the server has GOOD hardware components(16-32 Gb RAM;Xeon or quadcore........)
i keep having these notifications everyday; i searched on the internet and i found the cause to be 1 of the 2:
1) disk hardware issue( non common=rarely )
2) the queue time on the hard-disk( time to write on the Hard-disk)
if anyone can assist me with the following:
1) is this a serious issue that will affect our enviroment?
2) is it good to edit the time of monitoring to be 10minute(instead of the default 5min)
3) is there any solution for this?(to prevent these annoying -useless??--- notifications)
4)what is the cause of this queue delay;;and FYI sometime this happens when nothing and noone is using the server (i.e the server is almost Idle)
Regards

Answer 1

The problem is....  exactly what the knowledge of the alert says is wrong.  It is very simple.  Your disk latency is too high at times. 
This is likely due to overloading the capabilities of the disk, and during peak times, the disk is underperforming.  Or - it could be that occasionally, due to the design of your disks - you get a very large spike in disk latency... and this trips the
"average" counter.  You could change this monitor to be a consecutive sample threshold monitor, and that would likely quiet it down.... but only doing an analysis of a perfmon of several disks over 24 hours would you be able to determine specifically
whats going on.
SCOM did exactly what it is supposed to do.... it alerted your, proactively, to the possible existence of an issue.  Now you, using the knowledge already in the alert, use that information to further investigate, and determine what is the corrective
action to take. 
Summary
The Avg. Disk sec/Transfer (LogicalDisk\Avg. Disk sec/Transfer) for the logical disk has exceeded the threshold. The logical disk and possibly even overall system performance may significantly diminish which will result in poor operating system and application
performance.
The Avg. Disk sec/ Transfer counter measures the average rate of disk Transfer requests (I/O request packets (IRPs)) that are executed per second on a specific logical disk. This is one measure of storage subsystem throughput.
Causes
A high Avg. Disk sec/Transfer performance counter value may occur due to a burst of disk transfer requests by either an operating system or application.
Resolutions
To increase the available storage subsystem throughput for this logical disk, do one or more of the following:
•
Upgrade the controllers or disk drives.
•
Switch from RAID-5 to RAID-0+1.
•
Increase the number of actual spindles.
Be sure to set this threshold value appropriately for your specific storage hardware. The threshold value will vary according to the disk’s underlying storage subsystem. For example, the “disk” might be
a single spindle or a large disk array. You can use MOM overrides to define exception thresholds, which can be applied to specific computers or entire computer groups.
Additional Information
The Avg. Disk sec/Transfer counter is useful in gathering throughput data. If the average time is long enough, you can analyze a histogram of the array’s response to specific loads (queues, request sizes, and so on). If possible, you should
observe workloads separately.
You can use throughput metrics to determine:
•
The behavior of a workload running on a given host system. You can track the workload requirements for disk transfer requests over time. Characterization of workloads is an important part of performance analysis and capacity planning.
•
The peak and sustainable levels of performance that are provided by a given storage subsystem. A workload can either be used to artificially or naturally push a storage subsystem (in this case, a given logical disk) to its limits. Determining these
limits provides useful configuration information for system designers and administrators.
However, without thorough knowledge of the underlying storage subsystem of the logical disk (for example, knowing whether it is a single spindle or a massive disk array), it can be difficult to provide an optimized one size fits all threshold value.
You must also consider the Avg. Disk sec/Transfer counter in conjunction with other transfer request characteristics (for example, request size and randomness/sequentially) and the equivalent counters for write disk requests.
If the Avg. Disk sec/Transfers counter is tracked over time and if it increases with the intensity of the workloads that are driving the transfer requests, it is reasonable to suspect that the logical disk is saturated if throughput does not increase and
the user experiences degraded system throughput.
For more information about storage architecture and driver support, see the Storage - Architecture and Driver Support Web site at
http://go.microsoft.com/fwlink/?LinkId=26156.