Querying and Managing Provenance through User Views in Scientific Workflows

TLDR

Workflow systems increasingly chain bioinformatics tasks, generating large data volumes that demand reproducible results, so provenance is critical but often overwhelming, necessitating abstraction mechanisms. This work formalizes user views, demonstrates their use in provenance queries, and proposes an algorithm to generate views based on task relevance. User views define the sub‑workflow level visible to a user, determining which data and tasks appear in provenance queries; the authors implement a prototype and provide an algorithm for generating such views. The prototype yields performance results, and the user‑view technique generalizes to other data‑oriented workflows.

Abstract

Workflow systems have become increasingly popular for managing experiments where many bioinformatics tasks are chained together. Due to the large amount of data generated by these experiments and the need for reproducible results, provenance has become of paramount importance. Workflow systems are therefore starting to provide support for querying provenance. However, the amount of provenance information may be overwhelming, so there is a need for abstraction mechanisms to help users focus on the most relevant information. The technique we pursue is that of "user views". Since bioinformatics tasks may themselves be complex sub-workflows, a user view determines what level of sub-workflow the user can see, and thus what data and tasks are visible in provenance queries. In this paper, we formalize the notion of user views, demonstrate how they can be used in provenance queries, and give an algorithm for generating a user view based on which tasks are relevant for the user. We then describe our prototype and give performance results. Although presented in the context of scientific workflows, the technique applies to other data-oriented workflows.