GAN Dissection: Visualizing and Understanding Generative Adversarial\n Networks

Abstract

Generative Adversarial Networks (GANs) have recently achieved impressive\nresults for many real-world applications, and many GAN variants have emerged\nwith improvements in sample quality and training stability. However, they have\nnot been well visualized or understood. How does a GAN represent our visual\nworld internally? What causes the artifacts in GAN results? How do\narchitectural choices affect GAN learning? Answering such questions could\nenable us to develop new insights and better models.\n In this work, we present an analytic framework to visualize and understand\nGANs at the unit-, object-, and scene-level. We first identify a group of\ninterpretable units that are closely related to object concepts using a\nsegmentation-based network dissection method. Then, we quantify the causal\neffect of interpretable units by measuring the ability of interventions to\ncontrol objects in the output. We examine the contextual relationship between\nthese units and their surroundings by inserting the discovered object concepts\ninto new images. We show several practical applications enabled by our\nframework, from comparing internal representations across different layers,\nmodels, and datasets, to improving GANs by locating and removing\nartifact-causing units, to interactively manipulating objects in a scene. We\nprovide open source interpretation tools to help researchers and practitioners\nbetter understand their GAN models.\n