Journal of Information Policy · 2018 · 40 citations · 47 references
Public AffairsInformation SystemsPublic PolicyData EthicsDiscriminationStereotypingAlgorithmic FairnessAnti-discrimination PolicyBiasData AnonymizationData ProtectionEmployment DiscriminationAlgorithmic Bias In EmploymentBias DetectionBias MitigationDiscrimination DetectionDiscrimination In Automated Decision-makingFairnessSociotechnical SystemsSociotechnical Systems Analysis
Data‑driven models are widely used for high‑stakes decisions, and to safeguard privacy decision‑makers often omit social category data such as sex and race. The article argues that removing such data can worsen discrimination by obscuring bias. The authors explain that absent social category data, algorithms can generate or perpetuate biases, and demonstrate that incorporating this data reveals discriminatory patterns in education and employment, enabling targeted mitigation strategies. They conclude that discrimination can arise in any algorithmic decision system and outline implications for researchers and policymakers.
Abstract Organizations often employ data-driven models to inform decisions that can have a significant impact on people's lives (e.g., university admissions, hiring). In order to protect people's privacy and prevent discrimination, these decision-makers may choose to delete or avoid collecting social category data, like sex and race. In this article, we argue that such censoring can exacerbate discrimination by making biases more difficult to detect. We begin by detailing how computerized decisions can lead to biases in the absence of social category data and in some contexts, may even sustain biases that arise by random chance. We then show how proactively using social category data can help illuminate and combat discriminatory practices, using cases from education and employment that lead to strategies for detecting and preventing discrimination. We conclude that discrimination can occur in any sociotechnical system in which someone decides to use an algorithmic process to inform decision-making, and we offer a set of broader implications for researchers and policymakers.
47
Birds of a Feather: Homophily in Social Networks
Miller McPherson, Lynn Smith‐Lovin, James M. Cook · Annual Review of Sociology · 2001 · 18.6K citations
Michael Spence · The Quarterly Journal of Economics · 1973 · 15.2K citations
The Complexity of Intersectionality
Leslie McCall · Signs · 2005 · 6.5K citations