VMmark: A Scalable Benchmark for Virtualized Systems

Abstract

The increasing speed of computing resources coupled with the rise of robust and flexible virtual machine technology creates opportunities for consolidating multiple, variably loaded systems onto a single physical server. Traditional server benchmarks, which focus on a single workload, do not capture the system behavior induced by multiple virtual machines. An appropriate virtual machine benchmark should employ realistic, diverse workloads running on multiple operating systems. The benchmark should generate an easily understandable metric that scales with underlying system capacity using a controlled strategy based on a combination of increased individual workload scores and running an increasing number of workloads. This paper presents a tile-based benchmark consisting of several familiar workloads running simultaneously in separate virtual machines. Each workload component is based upon a single-system benchmark running at less than full utilization. This collection of different workloads is aggregated into a unit of work referred to as a tile. The performance of each workload is measured and used to form an aggregate score for the tile. The scores generated when running multiple tiles simultaneously may be summed to increase the overall benchmark score.