Measuring Semantic Similarity between Words Using Wikipedia

Abstract

Semantic similarity measures play an important role in the extraction of semantic relations. Semantic similarity measures are widely used in natural language processing (NLP) and information retrieval (IR). This paper presents a new Web-based method for measuring the semantic similarity between words. Different from other methods which are based on taxonomy or search engine in Internet, our method uses snippets from Wikipedia1 to calculate the semantic similarity between words by using cosine similarity and TF-IDF. Also, the stemmer algorithm and stop words are used in preprocessing the snippets from Wikipedia. We set different threshold to evaluate our results in order to decrease the interference from noise and redundancy. Our method was empirically evaluated using Rubenstein-Good enough benchmark dataset. It gives higher correlation value (with 0.615) than some existing methods. Evaluation results show that our method improves ac-curacy and more robust for measuring semantic similarity between words.