Proceedings of the VLDB Endowment · 2018 · 102 citations · 40 references
ComputingComputer ScienceData MiningDatabasesKnowledge RepresentationQuery ProcessingSearching AlgorithmsDependency DiscoveryDiscoveryFunctional Dependency Discovery
Functional dependencies (FDs) and unique column combinations (UCCs) form a valuable ingredient for many data management tasks, such as data cleaning, schema recovery, and query optimization. Because these dependencies are unknown in most scenarios, their automatic discovery has been well researched. However, existing methods mostly discover only exact dependencies, i.e., those without violations. Real-world dependencies, in contrast, are frequently approximate due to data exceptions, ambiguities, or data errors. This relaxation to approximate dependencies renders their discovery an even harder task than the already challenging exact dependency discovery. To this end, we propose the novel and highly efficient algorithm P yro to discover both approximate FDs and approximate UCCs. P yro combines a separate-and-conquer search strategy with sampling-based guidance that quickly detects dependency candidates and verifies them. In our broad experimental evaluation, P yro outperforms existing discovery algorithms by a factor of up to 33, scales to larger datasets, and at the same time requires the least main memory.
40
Mining frequent patterns without candidate generation
Jiawei Han, Jian Pei, Yiwen Yin · ACM SIGMOD Record · 2000 · 6.4K citations
A survey on feature selection methods
Girish Chandrashekar, Ferat Sahin · Computers & Electrical Engineering · 2013 · 5.1K citations
Mining frequent patterns without candidate generation
Jiawei Han, Jian Pei, Yiwen Yin · 2000 · 3.2K citations · Full text