A Classification System of Career Guidance Questions Based on Natural Language Processing and Supervised Machine Learning Techniques

Abstract

Questions classification is a crucial task in automatic question answering systems. In this paper, we present a system for classifying questions asked by students about higher studies and career choices based on their abilities and skills. The system uses natural language processing and machine learning techniques. We collected and labeled a dataset, preprocessed the textual content, and used the TF-IDF statistical representation to extract features. We implemented four supervised machine learning algorithms for classification: logistic regression (LR), decision tree (DT), K-nearest neighbors (KNN), and support vector machine (SVM). We also applied a feature selection technique to improve the system. The best accuracy achieved was 85.63% using the logistic regression classifier.