Distributed Storage and Querying Techniques for a Semantic Web of Scientific Workflow Provenance

Abstract

In scientific workflow environments, scientific discovery reproducibility, result interpretation, and problem diagnosis primarily depend on provenance, which records the history of an in-silico experiment. Resource Description Framework is frequently used to represent provenance based on vocabularies such as the Open Provenance Model. For complex scientific workflows that generate large amounts of RDF triples, single-machine provenance management becomes inadequate over time. In this paper, we research how HBase Bigtable-like capabilities can be leveraged for distributed storage and querying of provenance data represented in RDF. In particular, we architect the ProvBase system that incorporates an HBase/Hadoop backend, propose a storage schema to hold provenance triples, and design querying algorithms to evaluate SPARQL queries in the system. Using the Third Provenance Challenge queries, we conduct an experimental study to show the feasibility of our approach.