Software Requirements Classification Using Machine Learning Algorithms

TLDR

Accurate classification of software requirements is a critical task in software engineering. This study compares BoW, TF‑IDF, and CHI2 feature extraction methods and several machine learning algorithms to identify the most effective approach for classifying requirements into functional and non‑functional categories, using the PROMISE_exp dataset and aiming to provide a reference for the community. The authors processed the PROMISE_exp dataset with normalization, extracted features via BoW, TF‑IDF, and CHI2, and applied Logistic Regression, SVM, Multinomial Naïve Bayes, and k‑Nearest Neighbors for classification. TF‑IDF combined with Logistic Regression achieved the highest F‑measure of 0.91 for binary classification (tied with SVM), 0.74 for non‑functional classification, and 0.78 overall.

Abstract

The correct classification of requirements has become an essential task within software engineering. This study shows a comparison among the text feature extraction techniques, and machine learning algorithms to the problem of requirements engineer classification to answer the two major questions “Which works best (Bag of Words (BoW) vs. Term Frequency–Inverse Document Frequency (TF-IDF) vs. Chi Squared (CHI2)) for classifying Software Requirements into Functional Requirements (FR) and Non-Functional Requirements (NF), and the sub-classes of Non-Functional Requirements?” and “Which Machine Learning Algorithm provides the best performance for the requirements classification task?”. The data used to perform the research was the PROMISE_exp, a recently made dataset that expands the already known PROMISE repository, a repository that contains labeled software requirements. All the documents from the database were cleaned with a set of normalization steps and the two feature extractions, and feature selection techniques used were BoW, TF-IDF and CHI2 respectively. The algorithms used for classification were Logist Regression (LR), Support Vector Machine (SVM), Multinomial Naive Bayes (MNB) and k-Nearest Neighbors (kNN). The novelty of our work is the data used to perform the experiment, the details of the steps used to reproduce the classification, and the comparison between BoW, TF-IDF and CHI2 for this repository not having been covered by other studies. This work will serve as a reference for the software engineering community and will help other researchers to understand the requirement classification process. We noticed that the use of TF-IDF followed by the use of LR had a better classification result to differentiate requirements, with an F-measure of 0.91 in binary classification (tying with SVM in that case), 0.74 in NF classification and 0.78 in general classification. As future work we intend to compare more algorithms and new forms to improve the precision of our models.