The Earth System Grid: Supporting the Next Generation of Climate Modeling Research

TLDR

Understanding the Earth's climate system and its changing dynamics is a preeminent scientific challenge, with global climate models running on distributed supercomputers generating a rapidly growing archive of over 100 TB of data that is fundamentally a Grid problem. The study aims to anticipate and prepare for petabyte‑scale distributed climate research data holdings expected by mid‑decade. The Earth System Grid, built on the Globus toolkit and related technologies, provides authentication, authorization, large‑scale data transport, scalable replication, semantic cataloging, discovery, monitoring, and web portals to manage and access petabyte‑scale climate data across heterogeneous sites.

Abstract

Understanding the Earth's climate system and how it might be changing is a preeminent scientific challenge. Global climate models are used to simulate past, present, and future climates, and experiments are executed continuously on an array of distributed supercomputers. The resulting data archive, spread over several sites, currently contains upwards of 100 TB of simulation data and is growing rapidly. Looking toward mid-decade and beyond, we must anticipate and prepare for distributed climate research data holdings of many petabytes. The Earth System Grid (ESG) is a collaborative interdisciplinary project aimed at addressing the challenge of enabling management, discovery, access, and analysis of these critically important datasets in a distributed and heterogeneous computational environment. The problem is fundamentally a Grid problem. Building upon the Globus toolkit and a variety of other technologies, ESG is developing an environment that addresses authentication, authorization for data access, large-scale data transport and management, services and abstractions for high-performance remote data access, mechanisms for scalable data replication, cataloging with rich semantic and syntactic information, data discovery, distributed monitoring, and Web-based portals for using the system.