An empirical study of operating systems errors

TLDR

Unlike prior studies that rely on manual inspection, this work applies static analysis uniformly across the entire Linux and OpenBSD kernels, though it examines a narrower error set. The study investigates operating system errors detected via static compiler analysis of the Linux and OpenBSD kernels. The authors use automated static analysis to scan the entire kernel source across multiple releases, tracking errors over time to estimate their persistence before resolution. Device drivers exhibit three to seven times higher error rates than the rest of the kernel, the top quartile of functions shows two to six times higher rates than the bottom quartile, newer files have up to twice the error rate of older ones—indicating code hardening—and bugs persist on average 1.8 years before being fixed.

Abstract

We present a study of operating system errors found by automatic, static, compiler analysis applied to the Linux and OpenBSD kernels. Our approach differs from previous studies that consider errors found by manual inspection of logs, testing, and surveys because static analysis is applied uniformly to the entire kernel source, though our approach necessarily considers a less comprehensive variety of errors than previous studies. In addition, automation allows us to track errors over multiple versions of the kernel source to estimate how long errors remain in the system before they are fixed.We found that device drivers have error rates up to three to seven times higher than the rest of the kernel. We found that the largest quartile of functions have error rates two to six times higher than the smallest quartile. We found that the newest quartile of files have error rates up to twice that of the oldest quartile, which provides evidence that code "hardens" over time. Finally, we found that bugs remain in the Linux kernel an average of 1.8 years before being fixed.