Automated severity assessment of software defect reports

TLDR

In mission‑critical systems such as those developed by NASA, accurate severity assessment of defect reports is crucial for resource allocation and testing planning, yet it is heavily influenced by engineers’ experience and time spent on each issue. The study introduces SEVERIS, an automated tool that assists test engineers in assigning severity levels to defect reports. SEVERIS applies standard text‑mining and machine‑learning techniques to existing defect report datasets and was evaluated in a case study using NASApsilas Project data from the Issue Tracking System. The case study shows that SEVERIS reliably predicts severity levels, is easy to use, and operates efficiently.

Abstract

In mission critical systems, such as those developed by NASA, it is very important that the test engineers properly recognize the severity of each issue they identify during testing. Proper severity assessment is essential for appropriate resource allocation and planning for fixing activities and additional testing. Severity assessment is strongly influenced by the experience of the test engineers and by the time they spend on each issue. The paper presents a new and automated method named SEVERIS (severity issue assessment), which assists the test engineer in assigning severity levels to defect reports. SEVERIS is based on standard text mining and machine learning techniques applied to existing sets of defect reports. A case study on using SEVERIS with data from NASApsilas Project and Issue Tracking System (PITS) is presented in the paper. The case study results indicate that SEVERIS is a good predictor for issue severity levels, while it is easy to use and efficient.