Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice,\n and Discrimination

Abstract

To partly address people's concerns over web tracking, Google has created the\nAd Settings webpage to provide information about and some choice over the\nprofiles Google creates on users. We present AdFisher, an automated tool that\nexplores how user behaviors, Google's ads, and Ad Settings interact. AdFisher\ncan run browser-based experiments and analyze data using machine learning and\nsignificance tests. Our tool uses a rigorous experimental design and\nstatistical analysis to ensure the statistical soundness of our results. We use\nAdFisher to find that the Ad Settings was opaque about some features of a\nuser's profile, that it does provide some choice on ads, and that these choices\ncan lead to seemingly discriminatory ads. In particular, we found that visiting\nwebpages associated with substance abuse changed the ads shown but not the\nsettings page. We also found that setting the gender to female resulted in\ngetting fewer instances of an ad related to high paying jobs than setting it to\nmale. We cannot determine who caused these findings due to our limited\nvisibility into the ad ecosystem, which includes Google, advertisers, websites,\nand users. Nevertheless, these results can form the starting point for deeper\ninvestigations by either the companies themselves or by regulatory bodies.\n