SCRUPLES: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes

TLDR

AI systems increasingly permeate daily life, making it crucial that they grasp human ethical norms. The study aims to develop a data‑driven machine‑ethics approach by creating SCRUPLES, a large dataset of 625,000 judgments on 32,000 real‑life anecdotes, and by proposing a method that estimates optimal performance on tasks with diverse label distributions. SCRUPLES consists of 32,000 real‑life anecdotes with community judgment distributions, and the authors introduce a technique that separates intrinsic from model uncertainty to estimate best possible performance on such diverse‑label tasks. Neural language models struggle on SCRUPLES but perform better on simplified scenarios, indicating they can learn basic ethical building blocks, while the analysis shows that many situations are inherently divisive and norms are not always clear‑cut. Data and code are available at https://github.com/allenai/scruples.

Abstract

As AI systems become an increasing part of people's everyday lives, it becomes ever more important that they understand people's ethical norms. Motivated by descriptive ethics, a field of study that focuses on people's descriptive judgments rather than theoretical prescriptions on morality, we investigate a novel, data-driven approach to machine ethics. We introduce SCRUPLES, the first large-scale dataset with 625,000 ethical judgments over 32,000 real-life anecdotes. Each anecdote recounts a complex ethical situation, often posing moral dilemmas, paired with a distribution of judgments contributed by the community members. Our dataset presents a major challenge to state-of-the-art neural language models, leaving significant room for improvement. However, when presented with simplified moral situations, the results are considerably more promising, suggesting that neural models can effectively learn simpler ethical building blocks. A key take-away of our empirical analysis is that norms are not always clean-cut; many situations are naturally divisive. We present a new method to estimate the best possible performance on such tasks with inherently diverse label distributions, and explore likelihood functions that separate intrinsic from model uncertainty. Data and code are available at https://github.com/allenai/scruples.