"What Parts of Your Apps are Loved by Users?" (T)

Abstract

Recently, Begel et al. found that one of the most important questions software developers ask is "what parts of software are used/loved by users." User reviews provide an effective channel to address this question. However, most existing review summarization tools treat reviews as bags-of-words (i.e., mixed review categories) and are limited to extract software aspects and user preferences. We present a novel review summarization framework, SUR-Miner. Instead of a bags-of-words assumption, it classifies reviews into five categories and extracts aspects for sentences which include aspect evaluation using a pattern-based parser. Then, SUR-Miner visualizes the summaries using two interactive diagrams. Our evaluation on seventeen popular apps shows that SUR-Miner summarizes more accurate and clearer aspects than state-of-the-art techniques, with an F1-score of 0.81, significantly greater than that of ReviewSpotlight (0.56) and Guzmans' method (0.55). Feedback from developers shows that 88% developers agreed with the usefulness of the summaries from SUR-Miner.