2010 · 27 citations · 17 references
ComputingComputer ScienceComputer SystemsData ManagementDistributed SystemsGrid ComputingData ProvenanceDistributed DatabasesData-intensive ComputingDistributed ProvenanceGranularityMetadataProvenance CaptureProvenance QueryingProvenance RecordsProvenance Tracking
Current provenance collection systems typically gather metadata on remote hosts and submit it to a central server. In contrast, several data-intensive scientific applications require a decentralized architecture in which each host maintains an authoritative local repository of the provenance metadata gathered on that host. The latter approach allows the system to handle the large amounts of metadata generated when auditing occurs at fine granularity, and allows users to retain control over their provenance records. The decentralized architecture, however, increases the complexity of auditing, tracking, and querying distributed provenance. We describe a system for capturing data provenance in distributed applications, and the use of provenance sketches to optimize subsequent data provenance queries. Experiments with data gathered from distributed workflow applications demonstrate the feasibility of a decentralized provenance management system and improvements in the efficiency of provenance queries.
17
Network Applications of Bloom Filters: A Survey
Andrei Broder, Michael Mitzenmacher · Internet Mathematics · 2004 · 2K citations · Full text
A survey of data provenance in e-science
Yogesh Simmhan, Beth Plale, Dennis Gannon · ACM SIGMOD Record · 2005 · 1.2K citations
Chimera: a virtual data system for representing, querying, and automating data derivation
Ian Foster, Jens Vöckler, Michael Wilde et al. · 2002 · 609 citations
Trio: A system for data, uncertainty, and lineage
2006 · 329 citations