Model Compression with Adversarial Robustness: A Unified Optimization\n Framework

Abstract

Deep model compression has been extensively studied, and state-of-the-art\nmethods can now achieve high compression ratios with minimal accuracy loss.\nThis paper studies model compression through a different lens: could we\ncompress models without hurting their robustness to adversarial attacks, in\naddition to maintaining accuracy? Previous literature suggested that the goals\nof robustness and compactness might sometimes contradict. We propose a novel\nAdversarially Trained Model Compression (ATMC) framework. ATMC constructs a\nunified constrained optimization formulation, where existing compression means\n(pruning, factorization, quantization) are all integrated into the constraints.\nAn efficient algorithm is then developed. An extensive group of experiments are\npresented, demonstrating that ATMC obtains remarkably more favorable trade-off\namong model size, accuracy and robustness, over currently available\nalternatives in various settings. The codes are publicly available at:\nhttps://github.com/shupenggui/ATMC.\n