ACM SIGMOD Record · 2007 · 195 citations · 7 references
ComputingComputer ScienceData MiningDatabasesInformation RetrievalQuery ProcessingText Similarity AnalysisCosine SimilarityDocument SimilarityDocument Similarity MetricsEfficient EstimationSimilarity Search
An increasing number of database applications today require sophisticated approximate string matching capabilities. Examples of such application areas include data integration and data cleaning. Cosine similarity has proven to be a robust metric for scoring the similarity between two strings, and it is increasingly being used in complex queries. An immediate challenge faced by current database optimizers is to find accurate and efficient methods for estimating the selectivity of cosine similarity predicates. To the best of our knowledge, there are no known methods for this problem. In this paper, we present the first approach for estimating the selectivity of tf.idf based cosine similarity predicates. We evaluate our approach on three different real datasets and show that our method often produces estimates that are within 40% of the actual selectivity.
7
<i>Probability, Random Variables, and Stochastic Processes</i>
A. Papoulis, Joseph G. Hoffman · Physics Today · 1967 · 4.1K citations
Text joins in an RDBMS for web data integration
Luis Gravano, Panagiotis G. Ipeirotis, Nick Koudas et al. · 2003 · 170 citations
Using q-grams in a DBMS for Approximate String Processing.
Luis Gravano, Panagiotis G. Ipeirotis, H. V. Jagadish et al. · 2001 · 114 citations
Text joins for data cleansing and integration in an RDBMS
Luis Gravano, Panagiotis G. Ipeirotis, Nick Koudas et al. · 2004 · 57 citations