HoloClean

TLDR

HoloClean integrates qualitative integrity‑constraint‑based repair with quantitative statistical methods. The authors present HoloClean, a probabilistic inference–driven framework for holistic data repair, and propose optimizations that enable scaling to millions of tuples. HoloClean constructs a probabilistic program from the input data and applies inference optimizations to repair inconsistencies. HoloClean achieves about 90 % precision and 76 % recall, yielding a two‑fold F1 improvement over state‑of‑the‑art methods.

Abstract

We introduce HoloClean, a framework for holistic data repairing driven by probabilistic inference. HoloClean unifies qualitative data repairing, which relies on integrity constraints or external data sources, with quantitative data repairing methods, which leverage statistical properties of the input data. Given an inconsistent dataset as input, HoloClean automatically generates a probabilistic program that performs data repairing. Inspired by recent theoretical advances in probabilistic inference, we introduce a series of optimizations which ensure that inference over HoloClean's probabilistic model scales to instances with millions of tuples. We show that HoloClean finds data repairs with an average precision of ∼ 90% and an average recall of above ∼ 76% across a diverse array of datasets exhibiting different types of errors. This yields an average F1 improvement of more than 2× against state-of-the-art methods.