The effects of example-based explanations in a machine learning interface

TLDR

The black‑box nature of machine learning algorithms can make their predictions difficult to understand and explain to end‑users. In this paper, we propose and evaluate two kinds of example‑based explanations—normative and comparative—that automatically surface training‑set examples for a deep neural net sketch‑recognition algorithm. To investigate their effects, we deployed these explanations to 1,150 users on QuickDraw, an online platform where users draw images and see whether a recognizer has correctly guessed the intended drawing. When the algorithm failed to recognize a drawing, normative explanations improved users’ understanding and perceived system capability, whereas comparative explanations sometimes exposed limitations and did not consistently enhance perceptions, highlighting trade‑offs in example‑based explanations.

Abstract

The black-box nature of machine learning algorithms can make their predictions difficult to understand and explain to end-users. In this paper, we propose and evaluate two kinds of example-based explanations in the visual domain, normative explanations and comparative explanations (Figure 1), which automatically surface examples from the training set of a deep neural net sketch-recognition algorithm. To investigate their effects, we deployed these explanations to 1150 users on QuickDraw, an online platform where users draw images and see whether a recognizer has correctly guessed the intended drawing. When the algorithm failed to recognize the drawing, those who received normative explanations felt they had a better understanding of the system, and perceived the system to have higher capability. However, comparative explanations did not always improve perceptions of the algorithm, possibly because they sometimes exposed limitations of the algorithm and may have led to surprise. These findings suggest that examples can serve as a vehicle for explaining algorithmic behavior, but point to relative advantages and disadvantages of using different kinds of examples, depending on the goal.