Recently InnoTek, a company from southern Germany, released their virtualization software "VirtualBox" under GPL. Due this piece of software is based on qemu I thought about performing a benchmark to compare those two open source vmware competitors.
As some people requested, I added the VMware-Player to this article to make a direct comparison of those 2 opensource projects with this freeware tool.

Installation

My Host is running Fedora Core 6.
To install qemu is quite simple, because it's in the yum repository, so it's one command to get it working. The installation of Virtualbox is also very easy, just downloading and executing the installer from virtualbox.
It was really hard to get VMware working correctly on my system. First I tried to install the free VMware-Server edition, but after some hours of fiddling around and reading different tutorials, I just gave up and switched to the VMware-Player edition. But the installation of this piece of software didn't work out of the box too. I had to install it, create an empty config.h file in my kernel-headers, patch the whole thing with an update, which does not seem for me to be officially published by VMware, in order to get it finally working. VMware guys: bad job so far! If I had to pay for it, I would be really disappointed from this installation process!

Kernelmodules

Both, Qemu and VirtualBox come with support of a kernel module to directly execute the code of the guestsystem on the hostsystem.

• Qemu: kqemu - a closed source properitary product with a license that allows free useage for personal and commercial uses. Has to be dowlnoaded separatly from the qemu page.
• VirtualBox: vboxdrv - this so called "ring-0 support driver" is completely under gpl and it's already included in the installation package.

The installation procedures of the kernelmodules are very identic, you have to compile the module for your running kernel, and load the module using modprobe. For distros using udev you have to create a udef rule to set the permissions to the /dev/ file.

VMware-Player comes also with different kernel modules, which are compiled for the running kernel by the setup script, which is needed to configure the VMware installation.

Hardware

The host consists of following hardware:
Cpu: AMD Athlon(tm) XP 3000+ (2162 MHz)
Ram: 1024 MB
Harddisk: 160GB IDE

Configuration

VirtualBox comes with a qt gui to configure, manage and run the guestsystems, thus it is very intuitive to configure the guestsystem, also for a beginner. Additionally the package includes commandline tools, which are as powerful as the gui, what makes it interessting for the hardcore commandline users ;)
To configure the virtual system of qemu you have to go through the list of startup arguments and start it with the parameters which fit your needs.
VMware-Player does not bring any possibility to configure the guest system. This has to be done by an opensource windows tool called "vmmanager" which runs fine using wine.

Qemu configuration:


VirtualBox configuration:

Hardware of the guest

The guest hardware is configured like this:
Cpu: due to the kernelmodules, the cpu of the host is used, no simulation here
Ram: 256 MB
Video Card:

• VirtualBox: 8mb - VirtualBox videocard
• QEmu: 4mb - Cirrus Logic
• VMware: Unknown

Guestsoftware

On all virtual machines is following software installed:

• Windows 2000
• Free-AV

On the image running in VirtualBox are the "VirtualBox Guest Additions" installed, which include gfx and mouse drivers.
The VMware-player does not include the guest drivers for windows, thus Windows 2000 is not able to run the videocard using any higher resolution than 640x480 with 16 Colors.

Qemu:


VirtualBox:

Benchmark

I used the Windows tool "FreshDiagnose" from FreshDevices to benchmark the two systems. To ensure that nothing falsifies the results, I killed all running applications on the hostsystem and rebooted the virtual machine.

	VirtualBox	Qemu	VMware-player
CPU:
DhryStone ALU (MDIPS)	5,716	5,988	5,711
WhetStone FPU (MWIPS)	4,189	4,649	4,401
Multimedia Benchmark	2,152	268	2,071
Memory:
Integer Assignment	13,074	13,640	12,502
Real Assignment	13,782	14,270	14,176
Integer Split	18,027	19,554	18,885
Real Split	17,555	18,682	18,407
GFX:
Circles (Circles/s)	2,415	1,997	n/a
Rectangles (Rectangles)	3,123	6,548	n/a
Texts (Chars/s)	62,551	35,010	n/a
Harddisk:
Write Speed (MB/s)	8.65	12.44	6.07
Read Speed (MB/s)	14.65	23.22	4.68
Network:
Down Speed (MB/s)	4.9	2	5.3
Up Speed (MB/s)	5.2	2.3	4.1

Conclusions

VMware-Player does not seem to be that good and no real competitor to Virtualbox or Qemu after this benchmark and the Windows system running in the VMware-player didn't feel very smooth at all. That might be different, when the guest drivers are installed, but seems like they are only available with the commercial product.
Qemu and Virtualbox work great, Virtualbox feels much more smooth, but this might be mainly related to the "Guest Addition" drivers what makes the virtual machine feel like it would run on the physical hardware, e.g. the mouse doesn't lag like it does for Qemu.
Qemu is little bit faster in writing and reading files from/to the virtual harddisk, but the speed of the simulated network communication is very slow. Thus the communication between host and guest is very slow. A big advantage of Qemu is the support of multiple target architectures like powerpc, arm, mipsel and many more which are not supported by VirtualBox.

In summary:
VMware-Player did not meet the expectations I have to a professional product and I hope the commercial editions have a working installation process and make the guest operating system run more smooth.
I used Qemu wiht kqemu to run Windows as a virtual machine for a very long time period, but VirtualBox seems to be little bit better for this use case and the kernelmodule is also GPL, so I think it is time to switch.

But nevertheless the Qemu team and Innotek made really good work with those two projects. Thanks!