Explanations for human-on-the-loop

TLDR

Self‑adaptive systems often rely on human operators for expertise and problem detection, yet the benefits of placing humans on the loop and providing explanations are uncertain due to potential delays and erroneous judgments. This study introduces a formal framework to reason about when explanations of adaptive system behavior are warranted and how they can enhance human‑on‑the‑loop interaction. The framework defines explanations by content, effect, and cost, and employs a probabilistic dynamic adaptation method to decide when to present explanations to maximize overall system utility.

Abstract

Many self-adaptive systems benefit from human involvement and oversight, where a human operator can provide expertise not available to the system and can detect problems that the system is unaware of. One way of achieving this is by placing the human operator on the loop - i.e., providing supervisory oversight and intervening in the case of questionable adaptation decisions. To make such interaction effective, explanation is sometimes helpful to allow the human to understand why the system is making certain decisions and calibrate confidence from the human perspective. However, explanations come with costs in terms of delayed actions and the possibility that a human may make a bad judgement. Hence, it is not always obvious whether explanations will improve overall utility and, if so, what kinds of explanation to provide to the operator. In this work, we define a formal framework for reasoning about explanations of adaptive system behaviors and the conditions under which they are warranted. Specifically, we characterize explanations in terms of explanation content, effect, and cost. We then present a dynamic adaptation approach that leverages a probabilistic reasoning technique to determine when the explanation should be used in order to improve overall system utility.