Oracle VM Guest Performance as HVM

Question

All,
I was hoping to get clarification on this:
I happen to have a server with a processor that supports virtual machine in hardware (AMD-V). I have installed a couple of 64-bit OEL5u4 VMs off of the install iso (not from eDelivery templates, but from eDelivery iso). My guess is that these guests think they are using the hardware directly, which, as I undertand gets to be finally handled by qemu-dm in Dom0. Is this an efficient way of doing things with these OEL versions, or should I consider altering the guests so that they can communicate with Dom0 in a more efficient manner? The Oracle web site offers rpms for the guests, but seem to be targetted at OEL5 Update 1. Maybe these improvements are built in OELu4?
FWIW, I ran "hdparm -tT /dev/sda" on the host, and "hdparm -tT /dev/hda" in the guest, and the buffered read performance is worse in the VM by a factor of 5. The way I configured it, hda in the guest maps to a Logical Volume block device on the host, so I was hoping to get much better performance.
Any input welcome.

Answer 1

FWIW, I pursued this by reading the Oracle Server manual again, as well as what I could find from RHEL5. Again, this is trying to configure a guest with paravirtual drivers when the guest is run in full hardware virtualization, in order to retain as much disk performance as possible.
The short of it is that if the vm.vfg file makes use of "xvda" instead of "hda", then OEL5.4 uses the disk in a much more efficient manner (meaning it does a hypercall from DomU to Dom0, so Dum0 stops using the slower emulation in Dom0).
In my case, I chose to map the DomU hard disk to a Logical Volume in Dom0:
disk = [
'phy:/dev/DomUDisk2/LVOracleApp,xvda,w',
In this case, running "hdparm -tT :/dev/DomUDisk2/LVOracleApp" in Dom0 yields to near identical results to running "hdparm -tT /dec/xvda" in DomU. I used to have a loss of a factor 5 with emulation.
If anybody has to retrace my steps and try to replicate this, there are guides out there that mention installing missing paravirtual driver modules in the guest by downloading some rpm. Starting at OEL5.4, they are already there (/lib/modules/2.6.18-164.el5/kernel/drivers/xenpv_hvm/), so no extra rpms are required.
There is one caveat: Booting from xvda using the paravirtual drivers is somewhat less than automatic. Contrary to what you can read, grub does not seem to have a problem locating the kernel and initial ram disk. So, after grub does its thing and gives control to the kernel, the init script in the ram disk tries and locate the root file system. It then scans and finds the hard disk as hda (emulation), eventually mounts it, and runs init off of the disk. This new init has the paravirtual drivers from the hard disk, and finds the SAME DISK as xvda. This cannot be good, although it works and does not seem to thrash the file system as one would think. In order to prevent this from happening, there are a couple of changes to do in the grub configuration file /boot/grub/grub.cfg). The idea is to prevent the kernel from scanning for IDE disks:
kernel /vmlinuz-2.6.18-164.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet hda=noprobe ide0=noprobe
Now, once this is done, we have prevented the kernel from finding the root file system, and this VM is unbootable! The problem is that the initial ram drive does not have the paravirtual drivers built-in, so xvda, with the root file system, cannot be found. Fortunately, adding the paravirtual drivers to the initial ram drive is undramatic:
mkinitrd -f newinitrd 2.6.18-164.el5 --omit-scsi-modules with=xen-vbd with=xen-vnif preload xen-vbd preload xen-vnif*
The newinitrd file can then be copied to /boot with the right name for grub.
I thought I would pass this along as I found that there were great benefits to running the DomUs this way, performance-wise. At this point, I am not sure how these configurations would survive an upgrade of the kernel. I would think that an update to the initrd config file is in order. Thoughts welcome.