What's sex got to do with machine learning?

Abstract

The debate about fairness in machine learning has largely centered around competing substantive definitions of what fairness or nondiscrimination between groups requires. However, very little attention has been paid to what precisely a group is. Many recent approaches have abandoned observational, or purely statistical, definitions of fairness in favor of definitions that require one to specify a causal model of the data generating process. The implicit ontological assumption of these exercises is that a racial or sex group is a collection of individuals who share a trait or attribute, for example: the group "female" simply consists in grouping individuals who share female-coded sex features. We show this by exploring the formal assumption of modularity in causal models using directed acyclic graphs (DAGs), which hold that the dependencies captured by one causal pathway are invariant to interventions on any other causal pathways. Modeling sex, for example, as a node in a causal model aimed at elucidating fairness questions proposes two substantive claims: 1) There exists a feature, sex-on-its-own, that is an inherent trait of an individual that then (causally) brings about social phenomena external to it in the world; and 2) the relations between sex and its downstream effects can be modified in whichever ways and the former node would still retain the meaning that sex has in our world. Together, these claims suggest sex to be a category that could be different in its (causal) relations with other features of our social world via hypothetical interventions yet still mean what it means in our world. This fundamental stability of categories and causes (unless explicitly intervened on) is essential in the methodology of causal inference, because without it, causal operations can alter the meaning of a category, fundamentally change how it is situated within a causal diagram, and undermine the validity of any inferences drawn on the diagram as corresponding to any real phenomena in the world.