Untangling mixed information to calibrate resource utilization in virtual machines

Abstract

Server virtualization brings benefits in autonomic resource management, but also leads to new challenges. The challenge the paper addresses is on profiling physical resource utilization information of VMs when consolidated on a single server. Profiling is very difficult due to dynamic mapping relationships of resource activities between the virtual layer and the physical layer. The problem is further exacerbated by cross-resource utilization causality relationships due to virtualization overhead and resource utilization multiplexing across different VMs. We formulate profiling as a source separation problem as studied in digital signal processing, and design a directed factor graph (DFG) to model the multivariate dependence relationships among different resources (CPU, memory, disk, network) across virtual and physical layers. A benchmark based methodology is designed to build a DFG based model for the VM information calibration problem. A run-time calibration mechanism is proposed based on the DFG based model and further enhanced with a robust remodeling method based on guided regression. The proposed methodology outputs estimates of physical resource utilization on individual VMs and physical server aggregate resource utilization. We present a case study using the Xen-virtualization platform and evaluate the methodology for different consolidation scenarios with diverse applications including RUBiS, IOzone, SysBench, and Netperf. The results show that the DFG calibration significantly improves the accuracy of the resource utilization information collected within guest VMs as it reduces relative errors in CPU utilization from 44.8% down to 3.9% for CPU-intensive applications and relative errors in disk write rate from 391.5% down to 10.6% for IO-intensive applications, strongly arguing for the effectiveness of the proposed DFG calibration methodology.