Finding bugs is easy

TLDR

Software bug‑finding techniques, often based on formal methods and sophisticated analysis, exist but are hard to apply and sometimes ineffective, especially for bug patterns—common code idioms that are frequently erroneous. The paper aims to demonstrate the use of bug‑pattern detectors to uncover serious bugs in popular Java applications and libraries. The authors applied automatic bug‑pattern detectors to several widely used Java applications and libraries. The study shows that implementing bug‑pattern detectors is low effort and that even simple detectors uncover numerous bugs in real Java programs, revealing that well‑tested code can still contain obvious errors and that language features are often misused, yet straightforward techniques can mitigate these issues.

Abstract

Many techniques have been developed over the years to automatically find bugs in software. Often, these techniques rely on formal methods and sophisticated program analysis. While these techniques are valuable, they can be difficult to apply, and they aren't always effective in finding real bugs. Bug patterns are code idioms that are often errors. We have implemented automatic detectors for a variety of bug patterns found in Java programs. In this paper, we describe how we have used bug pattern detectors to find serious bugs in several widely used Java applications and libraries. We have found that the effort required to implement a bug pattern detector tends to be low, and that even extremely simple detectors find bugs in real applications.From our experience applying bug pattern detectors to real programs, we have drawn several interesting conclusions. First, we have found that even well tested code written by experts contains a surprising number of obvious bugs. Second, Java (and similar languages) have many language features and APIs which are prone to misuse. Finally, that simple automatic techniques can be effective at countering the impact of both ordinary mistakes and misunderstood language features.