It's Not Privacy, and It's Not Fair

Abstract

Classification is the foundation of targeting and tailoring information and experiences to individuals.Big data promises-or threatens-to bring classification to an increasing range of human activity.While many companies and government agencies foster an illusion that classification is (or should be) an area of absolute algorithmic rule-that decisions are neutral, organic, and even automatically rendered without human intervention-reality is a far messier mix of technical and human curating.Both the datasets and the algorithms reflect choices, among others, about data, connections, inferences, interpretation, and thresholds for inclusion that advance a specific purpose.Like maps that represent the physical environment in varied ways to serve different needs-mountaineering, sightseeing, or shopping-classification systems are neither neutral nor objective, but are biased toward their purposes.They reflect the explicit and implicit values of their designers.Few designers "see them as artifacts embodying moral and aesthetic choices" or recognize the powerful role they play in crafting "people's identities, aspirations, and dignity."1 But increasingly, the subjects of classification, as well as regulators, do.Today, the creation and consequences of some classification systems, from determination of tax-exempt status to predictive analytics in health insurance, from targeting for surveillance to systems for online behavioral advertising (OBA), are under scrutiny by consumer and data protection regulators,