Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks

TLDR

Collecting annotated image datasets is prohibitively expensive, so synthetic data is attractive but models trained solely on rendered images fail to generalize to real images, prompting prior unsupervised domain adaptation methods that map representations or learn domain‑invariant features. This study proposes an unsupervised approach that learns a pixel‑level transformation between source and target domains. The authors employ a generative adversarial network to adapt source‑domain images so they appear as if drawn from the target domain. The method generates realistic samples and surpasses state‑of‑the‑art performance on several unsupervised domain adaptation tasks, also generalizing to object classes unseen during training.

Abstract

Collecting well-annotated image datasets to train modern machine learning algorithms is prohibitively expensive for many tasks. One appealing alternative is rendering synthetic data where ground-truth annotations are generated automatically. Unfortunately, models trained purely on rendered images fail to generalize to real images. To address this shortcoming, prior work introduced unsupervised domain adaptation algorithms that have tried to either map representations between the two domains, or learn to extract features that are domain-invariant. In this work, we approach the problem in a new light by learning in an unsupervised manner a transformation in the pixel space from one domain to the other. Our generative adversarial network (GAN)-based method adapts source-domain images to appear as if drawn from the target domain. Our approach not only produces plausible samples, but also outperforms the state-of-the-art on a number of unsupervised domain adaptation scenarios by large margins. Finally, we demonstrate that the adaptation process generalizes to object classes unseen during training.