Scientific Programming · 2005 · 1.2K citations · 27 references
ComputingAstronomyComputer ScienceCluster ComputingDistributed SystemsScientific ComputingCluster ManagementDistributed ComputingDistributed Data ProcessingWorkflow DesignWorkflow ManagementWorkflow OrchestrationApplication Performance AssessmentApplication Performance OptimizationDistributed Workflow ExecutionDistributed Workflow ManagementRestructuring ImplementationScientific Workflow ManagementWorkflow Mapping
Scientific workflow mapping to distributed resources is challenging, and Pegasus offers a high‑level abstraction to simplify this process. This paper introduces Pegasus as a framework for mapping complex scientific workflows onto distributed resources. Pegasus represents workflows abstractly, allowing users to cluster tasks and deploy them on heterogeneous systems without specifying target details, as demonstrated on a real‑life astronomy application. Workflow restructuring with Pegasus clusters multiple tasks into single entities, yielding measurable performance improvements.
This paper describes the Pegasus framework that can be used to map complex scientific workflows onto distributed resources. Pegasus enables users to represent the workflows at an abstract level without needing to worry about the particulars of the target execution systems. The paper describes general issues in mapping applications and the functionality of Pegasus. We present the results of improving application performance through workflow restructuring which clusters multiple tasks in a workflow into single entities. A real‐life astronomy application is used as the basis for the study.
27
MPI: A Message-Passing Interface Standard
Message P Forum · 1994 · 2.7K citations
Grid information services for distributed resource sharing
Karl Czajkowski, S. Fitzgerald, Ian Foster et al. · 2002 · 1.6K citations
Condor-G: A Computation Management Agent for Multi-Institutional Grids
James Frey, Todd Tannenbaum, Miron Livny et al. · Cluster Computing · 2002 · 785 citations