Interpreting vmstat -i

Question

Hello,
I am trying to optimize network traffic on our firewall with multiple interfaces. I am trying to understand the output of vmstat -i and what it means.
Here is a sample output:
$ vmstat -i
interrupt total rate
clock 741511658 100
qfec0 825634725 111
qfec1 2441161380 329
qfec2 109965131 14
qfec3 0 0
qfec4 2245496826 302
qfec5 0 0
qfec6 0 0
qfec7 0 0
hmec0 217746015 29
Total 2286548439 308
Most of the traffic is between qfe4 and qfe1.
Some questions I am trying to answer are:
1. What does the rate value mean, and what is the context?
2. Would the system benefit from moving the qfe4 network to qfe3?
3. How do I determine if I am reaching hardware limitations (PCI bus contention, excessive interrupts, etc.?)
4. Would an additional CPU help?
Any help or pointers would be appreciated.
Thanks,
David Alten
[email protected]
Environment:
$ ./prtdiag
System Configuration: Sun Microsystems sun4u Sun Enterprise 220R (UltraSPARC-II 360MHz)
System clock frequency: 120 MHz
Memory size: 256 Megabytes
========================= CPUs =========================
Run Ecache CPU CPU
Brd CPU Module MHz MB Impl. Mask
0 0 0 360 4.0 US-II 10.0
========================= IO Cards =========================
Bus Freq
Brd Type MHz Slot Name Model
0 PCI 33 On-Board network-SUNW,hme
0 PCI 33 On-Board scsi-glm/disk (block) Symbios,53C875
0 PCI 33 On-Board scsi-glm/disk (block) Symbios,53C875
0 PCI 33 pcib slot 2 network-pci108e,2bad SUNW,pci-gem
0 PCI 33 pcib slot 2 SUNW,ifp-pci1077,2100.1.1077.+
0 PCI 33 pcib slot 4 SUNW,qfe-pci108e,1001 SUNW,pci-qfe
0 PCI 33 pcib slot 4 SUNW,qfe-pci108e,1001 SUNW,pci-qfe
0 PCI 33 pcib slot 4 SUNW,qfe-pci108e,1001 SUNW,pci-qfe
0 PCI 33 pcib slot 4 SUNW,qfe-pci108e,1001 SUNW,pci-qfe
0 PCI 33 pcib slot 5 TSI,gfxp GFXP
0 PCI 33 pcia slot 1 SUNW,qfe-pci108e,1001 SUNW,pci-qfe
0 PCI 33 pcia slot 1 SUNW,qfe-pci108e,1001 SUNW,pci-qfe
0 PCI 33 pcia slot 1 SUNW,qfe-pci108e,1001 SUNW,pci-qfe
0 PCI 33 pcia slot 1 SUNW,qfe-pci108e,1001 SUNW,pci-qfe
No failures found in System
===========================

Answer 1

Hi there,
If you want to do performance mesurement/tuning, you should check out SeToolkit
Rich Pettit and Adrian Cockcroft are the guru's, and have created these great tools!
http://www.setoolkit.com
some details on how to understand vmstat output:
r = Run queue length, the number of runnable processes that are not running
b = Processes blocked waiting for I/O, The number of blocked kernel threads waiting for resources, I/O paging and so forth. The number of processes waiting for a block device transfer (disk I/O to complete).
w = Idle processes that have been blocked for some time. The number of swapped out LWPs waiting for processing resources to finish; the swap queue length. the number of idle swapped out processes, not explained properly in the man pages since that measure used to be the number of active swapped-out processes waiting to be swapped back in.
swap = Free and unreserved swap available in Kbytes. The amount of available swap space currently available in Kbytes. If this number ever goes to zero, your system will hang and be unable to start any more processes.
free = Free memory measured in pages. The size of the free memory listed in Kbytes. The pages of RAM that are immediately ready to be used whenever a process starts up or needs more memory
re = pages reclaimed from the free list. The number of pages reclaimed from the free list . The page had been stolen from a process but had not yet been reused by a different process so it can be reclaimed by the process, thus avoiding a full page fault that would need I/O.
mf = Minor faults, The page was in memory but was not mapped. The number of minor faults in pages. A minor fault is caused by an address space or hardware address translation fault the can be resolved without performing a page-in. It is fast and uses only a little CPU time. Thus the process does not have to stop and wait, as long as a page is available for use from the free list.
pi = pages paged in from the file system or swap device, the number of Kbytes paged per second
po = pages paged out to the file system or swap device , the number of Kbytes paged per second
fr = pages that have been destroyed or freed The number of Kbytes freed per second . Pages freed is the rate at which memory is being put onto the freed list by the page scanner daemon. Pages are usually freed by the page scanner daemon but other mechanisms, such as processes exiting also free pages.
de = pages freed after writes, The deficit or the anticipated, short term memory shortfall needed by the recently swapped in processes. If the value is nonzero then memory was recently being consumed quickly and the extra free memory will be reclaimed in anticipation that it will be needed
sr = The number of pages scanned by the page daemon as it looks for pages to steal from the processes that are not using them often. This number is the key memory shortage indicator if it stays above 200 pages per second for long periods of time. This is the most important value
s0 = disk I/OS per second (may be multiple disks) the number of disk operations per second. There are slots for up to four disks, labeled with a single letter and number. The letter indicates the type of disk (s=scsi) IPI, and so forth: the number is the logical unit number
faults;
in = interrupts/second; The number of interrupts per second
sy = system calls per second; The number of system calls per second
cs = Process contact switches/second. The number of CPU context switches per second
CPU- A breakdown of percentage usage of CPU time. On multiprocessor systems, this is the average across all processors
us = user CPU time, The percentage of time that the processor is in user mode
sy = kernel CPU time, The percentage of time that the processor is in system mode
id = idle + wait CPU time, The percentage of time that the processor is idle.