Design of an Adaptive Framework for Utility-Based Optimization of Scientific Applications in the Cloud

Abstract

Cloud computing plays an increasingly important role in realizing scientific applications by offering virtualized compute and storage infrastructures that can scale on demand. In this paper we report on the design of a self-configuring adaptive framework for developing and optimizing scientific applications on top of Cloud technologies. Our framework relies on a MAPE-K loop, known from autonomic computing, for optimizing the configuration of scientific applications taking into account the three abstraction layers of the Cloud stack: the application layer, the execution environment layer, and the resource layer. By evaluating monitored resources, the framework configures the layers and allocates resources on a per job basis. The evaluation of configurations relies on historic data and a utility function that ranks different configurations regarding to the arising costs. The adaptive framework has been integrated into the Vienna Cloud Environment (VCE) and has been evaluated with a MapReduce application.