Empirical observation of negligible fairness-accuracy trade-offs in\n machine learning for public policy

Abstract

Growing use of machine learning in policy and social impact settings have\nraised concerns for fairness implications, especially for racial minorities.\nThese concerns have generated considerable interest among machine learning and\nartificial intelligence researchers, who have developed new methods and\nestablished theoretical bounds for improving fairness, focusing on the source\ndata, regularization and model training, or post-hoc adjustments to model\nscores. However, little work has studied the practical trade-offs between\nfairness and accuracy in real-world settings to understand how these bounds and\nmethods translate into policy choices and impact on society. Our empirical\nstudy fills this gap by investigating the impact of mitigating disparities on\naccuracy, focusing on the common context of using machine learning to inform\nbenefit allocation in resource-constrained programs across education, mental\nhealth, criminal justice, and housing safety. Here we describe applied work in\nwhich we find fairness-accuracy trade-offs to be negligible in practice. In\neach setting studied, explicitly focusing on achieving equity and using our\nproposed post-hoc disparity mitigation methods, fairness was substantially\nimproved without sacrificing accuracy. This observation was robust across\npolicy contexts studied, scale of resources available for intervention, time,\nand relative size of the protected groups. These empirical results challenge a\ncommonly held assumption that reducing disparities either requires accepting an\nappreciable drop in accuracy or the development of novel, complex methods,\nmaking reducing disparities in these applications more practical.\n