Query flocks

TLDR

Association‑rule mining succeeds because its objectives align with data and because query‑optimization tricks such as the a‑priori method accelerate processing. This paper extends those tricks to a broader setting, enabling efficient mining of very large databases for diverse pattern types. The proposed “query flocks” framework is a generate‑and‑test model that can be applied in general‑purpose mining systems or next‑generation query optimizers.

Abstract

Association-rule mining has proved a highly successful technique for extracting useful information from very large databases. This success is attributed not only to the appropriateness of the objectives, but to the fact that a number of new query-optimization ideas, such as the “a-priori” trick, make association-rule mining run much faster than might be expected. In this paper we see that the same tricks can be extended to a much more general context, allowing efficient mining of very large databases for many different kinds of patterns. The general idea, called “query flocks,” is a generate-and-test model for data-mining problems. We show how the idea can be used either in a general-purpose mining system or in a next generation of conventional query optimizers.