Learning to Generalize: Meta-Learning for Domain Generalization

TLDR

Domain shift causes models trained on one source domain to perform poorly on target domains with different statistics, and domain generalization methods aim to produce models that generalize well to novel testing domains. This work proposes a model‑agnostic meta‑learning procedure for domain generalization. The algorithm simulates train‑test domain shift by synthesizing virtual testing domains within each mini‑batch and optimizes a meta‑objective that requires improvements on training domains to translate to testing domains, thereby training models with strong generalization to unseen domains. The method achieves state‑of‑the‑art performance on a recent cross‑domain image classification benchmark and shows promise on two classic reinforcement learning tasks.

Abstract

Domain shift refers to the well known problem that a model trained in one source domain performs poorly when appliedto a target domain with different statistics. Domain Generalization (DG) techniques attempt to alleviate this issue by producing models which by design generalize well to novel testing domains. We propose a novel meta-learning method for domain generalization. Rather than designing a specific model that is robust to domain shift as in most previous DG work, we propose a model agnostic training procedure for DG. Our algorithm simulates train/test domain shift during training by synthesizing virtual testing domains within each mini-batch. The meta-optimization objective requires that steps to improve training domain performance should also improve testing domain performance. This meta-learning procedure trains models with good generalization ability to novel domains. We evaluate our method and achieve state of the art results on a recent cross-domain image classification benchmark, as well demonstrating its potential on two classic reinforcement learning tasks.