Provenance trails in the Wings/Pegasus system

TLDR

Large‑scale scientific workflows often involve thousands of computations over distributed, shared resources. The paper proposes a semantic‑based approach for creating, refining, and mapping scientific workflows to computing resources, aiming to support efficient execution and provenance tracking. Using semantic representations, the method automatically generates data‑independent workflows, maps them to available resources, and records provenance during creation and execution within the Wings/Pegasus system. The approach was implemented in Wings/Pegasus, demonstrated across multiple scientific domains, and successfully answered queries in the First Provenance Challenge. © 2007 John Wiley & Sons, Ltd.

Abstract

Abstract Our research focuses on creating and executing large‐scale scientific workflows that often involve thousands of computations over distributed, shared resources. We describe an approach to workflow creation and refinement that uses semantic representations to (1) describe complex scientific applications in a data‐independent manner, (2) automatically generate workflows of computations for given data sets, and (3) map the workflows to available computing resources for efficient execution. Our approach is implemented in the Wings/Pegasus workflow system and has been demonstrated in a variety of scientific application domains. This paper illustrates the application‐level provenance information generated Wings during workflow creation and the refinement provenance by the Pegasus mapping system for execution over grid computing environments. We show how this information is used in answering the queries of the First Provenance Challenge. Copyright © 2007 John Wiley & Sons, Ltd.