Dynamic virtual clusters in a grid site manager

TLDR

The paper introduces Cluster‑on‑Demand (COD), a dynamic resource‑management system that uses virtual clusters to enable advanced resource allocation in computing utilities and grids, demonstrated via Sun GridEngine experiments. COD implements dynamic virtual clusters by allocating servers from a shared pool and providing independent software environments, namespaces, access controls, and storage, while enabling policy‑based sharing, reservation, adaptive provisioning, idle‑resource scavenging, and dynamic Grid service instantiation through new cluster‑management functions that minimally affect existing Grid middleware.

Abstract

This paper presents new mechanisms for dynamic resource management in a cluster manager called Cluster-on-Demand (COD). COD allocates servers from a common pool to multiple virtual clusters (vclusters), with independently configured software environments, name spaces, user access controls, and network storage volumes. We present experiments using the popular Sun GridEngine batch scheduler to demonstrate that dynamic virtual clusters are an enabling abstraction for advanced resource management in computing utilities and grids. In particular, they support dynamic, policy-based cluster sharing between local users and hosted Grid services, resource reservation and adaptive provisioning, scavenging of the idle resources, and dynamic instantiation of Grid services. These goals are achieved in a direct and general way through a new set of fundamental cluster management functions, with minimal impact on the Grid middleware itself.