Nimrod/G: an architecture for a resource management and scheduling system in a global computational grid

TLDR

High‑performance computing on distributed systems is enabled by powerful microprocessors and high‑speed networks, yet geographically dispersed resources with varying ownership, access policies, and costs create significant integration and coordination challenges. Nimrod/G aims to manage and schedule computations over geographically dispersed, dynamic resources by leveraging a modular, component‑based architecture and computational‑economy‑driven scheduling on the Globus testbed. It implements this through a modular, component‑based design that utilizes GUSTO (GlobUS Toolkit) services and can be extended to other grid middleware.

Abstract

The availability of powerful microprocessors and high-speed networks as commodity components has enabled high-performance computing on distributed systems (wide-area cluster computing). In this environment, as the resources are usually distributed geographically at various levels (department, enterprise or worldwide), there is a great challenge in integrating, coordinating and presenting them as a single resource to the user, thus forming a computational grid. Another challenge comes from the distributed ownership of resources, with each resource having its own access policy, cost and mechanism. The proposed Nimrod/G grid-enabled resource management and scheduling system builds on our earlier work on Nimrod (D. Abramson et al., 1994, 1995, 1997, 2000) and follows a modular and component-based architecture enabling extensibility, portability, ease of development, and interoperability of independently developed components. It uses the GUSTO (GlobUS TOolkit) services and can be easily extended to operate with any other emerging grid middleware services. It focuses on the management and scheduling of computations over dynamic resources scattered geographically across the Internet at department, enterprise or global levels, with particular emphasis on developing scheduling schemes based on the concept of computational economy for a real testbed, namely the Globus testbed (GUSTO).