Man vs. machine: practical adversarial detection of malicious crowdsourcing workers

Abstract

Recent work in security and systems has embraced the use of machine learning (ML) techniques for identify-ing misbehavior, e.g. email spam and fake (Sybil) users in social networks. However, ML models are typically derived from fixed datasets, and must be periodically retrained. In adversarial environments, attackers can adapt by modifying their behavior or even sabotaging ML models by polluting training data. In this paper1, we perform an empirical study of ad-versarial attacks against machine learning models in the context of detecting malicious crowdsourcing systems, where sites connect paying users with workers willing to carry out malicious campaigns. By using human work-ers, these systems can easily circumvent deployed se-curity mechanisms, e.g. CAPTCHAs. We collect a dataset of malicious workers actively performing tasks on Weibo, China’s Twitter, and use it to develop ML-based detectors. We show that traditional ML techniques are accurate (95%–99%) in detection but can be highly vulnerable to adversarial attacks, including simple eva-sion attacks (workers modify their behavior) and power-ful poisoning attacks (where administrators tamper with the training set). We quantify the robustness of ML clas-sifiers by evaluating them in a range of practical adver-sarial models using ground truth data. Our analysis pro-vides a detailed look at practical adversarial attacks on ML models, and helps defenders make informed deci-sions in the design and configuration of ML detectors. 1