Algorithmic Fairness: Choices, Assumptions, and Definitions

TLDR

Recent research has sought to define fairness in machine‑learning predictions, but inconsistent terminology hampers comparison. The paper aims to bring order to the field by providing a concise reference for evaluating choices, assumptions, and fairness concerns in prediction‑based decision systems. The authors explicate implicit choices and assumptions underlying prediction‑based decisions, then demonstrate how these raise fairness concerns and compile a notationally consistent catalogue of fairness definitions from the ML literature.

Abstract

A recent flurry of research activity has attempted to quantitatively define "fairness" for decisions based on statistical and machine learning (ML) predictions. The rapid growth of this new field has led to wildly inconsistent terminology and notation, presenting a serious challenge for cataloguing and comparing definitions. This paper attempts to bring much-needed order. First, we explicate the various choices and assumptions made---often implicitly---to justify the use of prediction-based decisions. Next, we show how such choices and assumptions can raise concerns about fairness and we present a notationally consistent catalogue of fairness definitions from the ML literature. In doing so, we offer a concise reference for thinking through the choices, assumptions, and fairness considerations of prediction-based decision systems.