Towards supporting the life cycle of large scale scientific experiments

TLDR

Scientific experiments require managing and exchanging computational resources, and effective management demands facilities such as specification techniques, workflow heuristics, and provenance mechanisms that span composition, execution, and analysis phases; most work focuses only on execution and analysis, leaving the full life cycle unsupported, especially in large‑scale experiments. We propose an approach for managing large‑scale experiments based on provenance gathering during all phases of the life cycle. The method relies on continuous provenance collection across composition, execution, and analysis. The approach is expected to give scientists greater control over experiment trials.

Abstract

One of the main challenges of scientific experiments is to allow scientists to manage and exchange their scientific computational resources (data, programs, models, etc.). The effective management of such experiments requires a specific set of cardinal facilities, such as experiment specification techniques, workflow derivation heuristics and provenance mechanisms. These facilities may characterise the experiment life cycle into three phases: composition, execution, and analysis. Works concerned with supporting scientific workflows are mainly concerned with the execution and analysis phase. Therefore, they fail to support the scientific experiment throughout its life cycle as a set of integrated experimentation technologies. In large scale experiments this represents a research challenge. We propose an approach for managing large scale experiments based on provenance gathering during all phases of the life cycle. We foresee that such approach may aid scientists to have more control on the trials of the scientific experiment.