Anatomy of a failure

TLDR

The four themes—engagement, reference, accommodation, and surprise and insight—offer sensitivities that complement traditional task-based and open-ended evaluation approaches. This paper examines the failure of a novel sensor-based system designed to evoke user interpretation and appropriation in domestic settings by contrasting participant interactions with those from more successful deployments to identify symptoms of failure. The authors investigate failure causes at three levels: design hypothesis specificity, input-output mapping issues, and shortcomings in the design process employed. The system exhibited symptoms of failure across all four themes, demonstrating that interpretive systems can fail in clear, instructive ways.

Abstract

In this paper, we describe the failure of a novel sensor-based system intended to evoke user interpretation and appropriation in domestic settings. We contrast participants' interactions in this case study with those observed during more successful deployments to identify 'symptoms of failure' under four themes: engagement, reference, accommodation, and surprise and insight. These themes provide a set of sensitivities or orientations that may complement traditional task-based approaches to evaluation as well as the more open-ended ones we describe here. Our system showed symptoms of failure under each of these themes. We examine the reasons for this at three levels: problems particular to the specific design hypothesis; problems relevant for input-output mapping more generally; and problems in the design process we used. We conclude by noting that, although interpretive systems such as the one we describe here may succeed in a myriad of different ways, it is reassuring to know that they can also fail, and fail incontrovertibly, yet instructively.