2004 · 568 citations · 36 references
ComputingBioinformaticsComputer ScienceAlgorithmsData MiningMachine LearningParameterized AlgorithmsPattern RecognitionCompressionCompression AlgorithmsKolmogorov ComplexityKolmogorov Theory
Data‑mining algorithms typically rely on many tunable parameters, which can cause failure to detect true patterns or generate spurious results, motivating a shift toward parameter‑free methods that let the data speak. This study proposes a parameter‑free data‑mining paradigm inspired by recent bioinformatics and computational theory advances. Leveraging Kolmogorov complexity insights, the authors implement the approach with any compression algorithm plus minimal code, demonstrating competitive or superior performance in anomaly detection, classification, and clustering across time‑series, DNA, text, and video datasets.
Most data mining algorithms require the setting of many input parameters. Two main dangers of working with parameter-laden algorithms are the following. First, incorrect settings may cause an algorithm to fail in finding the true patterns. Second, a perhaps more insidious problem is that the algorithm may report spurious patterns that do not really exist, or greatly overestimate the significance of the reported patterns. This is especially likely when the user fails to understand the role of parameters in the data mining process.Data mining algorithms should have as few parameters as possible, ideally none. A parameter-free algorithm would limit our ability to impose our prejudices, expectations, and presumptions on the problem at hand, and would let the data itself speak to us. In this work, we show that recent results in bioinformatics and computational theory hold great promise for a parameter-free data-mining paradigm. The results are motivated by observations in Kolmogorov complexity theory. However, as a practical matter, they can be implemented using any off-the-shelf compression algorithm with the addition of just a dozen or so lines of code. We will show that this approach is competitive or superior to the state-of-the-art approaches in anomaly/interestingness detection, classification, and clustering with empirical tests on time series/DNA/text/video datasets.
36
Modeling by shortest data description
J. Rissanen · Automatica · 1978 · 6K citations
Data mining and knowledge discovery: making sense out of data
U.M. Feyyad · IEEE Expert · 1996 · 4.7K citations
A symbolic representation of time series, with implications for streaming algorithms
Jessica Lin, Eamonn Keogh, Stefano Lonardi et al. · 2003 · 1.8K citations
Christos Faloutsos, King-Ip Lin · 1995 · 901 citations · Full text