Study and Comparision of Vectorization Techniques Used in Text Classification

Abstract

Reviews on products and movies play an important role in predicting and formulating business strategies. Entertainment media, E-commerce, and social media use customers’ reviews to analyze customers’ requirements and level of satisfaction with the product. Business Analyst uses Sentiment Analysis for analyzing the attitude of the users from their reviews. E-commerce websites, entertainment and social media posts, tweets, comments, reviews, status, etc are the major sources of sentiment data (reviews). In the review system, users give the rating on a predefined scale of (1-5) i.e lowest to highest in terms of their satisfaction. As sentiment Analysis is one of the major applications of Machine Learning and machine learning deals with numeric data, so, textual-based review data needs to be converted into numeric data. Conversion of text to numeric form requires a large amount of memory and it is time-consuming also. This paper presents various vectorization techniques and their comparison in terms of memory management to convert text file into a vector file. The comparison shows gensim library-based Doc2Vec approach reduces memory requirements by up to 80%. This will also reduce the time consumption for task analysis and data processing of the model.