Disparate Impact in Big Data Policing

Abstract

Data-driven decision systems are taking over. Noinstitution in society seems immune from theenthusiasm that automated decision-making generates,including-and perhaps especially-the police. Policedepartments are increasingly deploying data miningtechniques to predict, prevent, and investigate crime.But all data mining systems have the potential foradverse impacts on vulnerable communities, andpredictive policing is no different. Determiningindividuals' threat levels by reference to commercialand social data can improperly link dark skin to higherthreat levels or to greater suspicion of havingcommitted a particularcrime. Crime mapping basedon historical data can lead to more arrests for nuisancecrimes in neighborhoods primarilypopulated by peopleof color. These effects are an artifact of the technologyitself, and will likely occur even assuming good faith onthe part of the police departments using it. Meanwhile,predictive policing is sold in part as a "neutral"methodto counteract unconscious biases when it is not simply\nsold to cash-strapped departments as a more cost-efficient way to do policing.The degree to which predictive policing systems havethese discriminatory results is unclear to the publicand to the police themselves, largely because there is noincentive in place for a department focused solely on"crime control" to spend resources asking the question.This is a problem for which existing law does notprovide a solution. Finding that neither the typicalconstitutional modes of police regulation nor ahypothetical anti-discriminationlaw would provide asolution, this Article turns toward a new regulatoryproposal centered on "algorithmicimpact statements."Modeled on the environmental impact statements ofthe National Environmental Policy Act, algorithmicimpact statements would require police departments toevaluate the efficacy and potential discriminatoryeffects of all available choices for predictive policingtechnologies. The regulation would also allow thepublic to weigh in through a notice-and-commentprocess. Such a regulation would fill the knowledgegap that makes future policy discussions about thecosts and benefits of predictive policing all butimpossible. Being primarily procedural, it would notnecessarily curtail a department determined todiscriminate, but by forcing departments to considerthe question and allowing society to understand thescope of the problem, it is a first step towards solvingthe problem and determining whether furtherintervention is require