Algorithms as partners in crime: A lesson in ethics by design

TLDR

Human‑in‑the‑loop approaches are widely promoted as a remedy for the ethical risks posed by increasingly autonomous AI systems. The study asks whether humans can effectively prevent unethical machine decisions. Online experiments were conducted in which either a machine corrected a human’s choice or a human corrected a machine’s choice. The results show that when humans correct machines they tend to exploit machine errors, whereas machine corrections produce similar decisions regardless of the corrective agent, suggesting that the human in the loop is not a reliable moral safeguard and that a design where humans decide and machines correct is preferable.

Abstract

The human in the loop is often advocated as a panacea against concerns about AI-powered machines, which increasingly take decisions of consequence in all realms of life. However, can we rely on humans to prevent unethical decisions by machines? We run online experiments modeling both the case where the machine serves as a corrective to the human and where the human serves as a corrective to the machine. Our results suggest that, in the former case, humans make similar decisions whether the corrective is a machine or another human. In the latter case, humans take advantage of rather than correct bad decisions by machines, turning into partners in crime. These findings caution us not to count too much on the human in the loop as a moral corrective. Instead, they tend to argue for human–machine decision-making where the human makes the decision and the machine is the corrective.