An empirical study on TensorFlow program bugs

TLDR

Deep learning is increasingly used in critical domains, yet defects can cause catastrophic failures and their characteristics have not yet been studied. The study collected TensorFlow program bugs from StackOverflow and GitHub to investigate their characteristics. The authors extracted bug information from QA pages, commits, pull requests, and issue discussions to analyze root causes, symptoms, and detection strategies. The results provide insights into TensorFlow coding defects and suggest new research directions.

Abstract

Deep learning applications become increasingly popular in important domains such as self-driving systems and facial identity systems. Defective deep learning applications may lead to catastrophic consequences. Although recent research efforts were made on testing and debugging deep learning applications, the characteristics of deep learning defects have never been studied. To fill this gap, we studied deep learning applications built on top of TensorFlow and collected program bugs related to TensorFlow from StackOverflow QA pages and Github projects. We extracted information from QA pages, commit messages, pull request messages, and issue discussions to examine the root causes and symptoms of these bugs. We also studied the strategies deployed by TensorFlow users for bug detection and localization. These findings help researchers and TensorFlow users to gain a better understanding of coding defects in TensorFlow programs and point out a new direction for future research.