Semantic Analysis Techniques using Twitter Datasets on Big Data: Comparative Analysis Study

Abstract

This paper conducts a comprehensive review of various word and sentence semantic similarity techniques proposed in the literature. Corpus-based, Knowledge-based, and Feature-based are categorized under word semantic similarity techniques. String and set-based, Word Order-based Similarity, POSbased, Syntactic dependency-based are categorized as sentence semantic similarity techniques. Using these techniques, we propose a model for computing the overall accuracy of the twitter dataset. The proposed model has been tested on the following four measures: Atish’s measure, Li’s measure, Mihalcea’s measure with path similarity, and Mihalcea’s measure with Wu and Palmer’s (WuP) similarity. Finally, we evaluate the proposed method on three real-world twitter datasets. The proposed model based on Atish’s measure seems to offer good results in all datasets when compared with the proposed model based on other sentence similarity measures.