Adaptation Regularization: A General Framework for Transfer Learning

TLDR

Domain transfer learning, which learns a target classifier using labeled data from a different distribution, has shown promise yet remains challenging, partly because prior work treated distribution adaptation and label propagation separately. This paper introduces Adaptation Regularization based Transfer Learning (ARTL), a unified framework that integrates distribution adaptation and label propagation under structural risk minimization and regularization theory. ARTL learns an adaptive classifier by jointly optimizing structural risk, aligning joint distributions across domains, and enforcing manifold consistency, with two instantiations based on Regularized Least Squares and Support Vector Machines derived via the representer theorem. Experiments on several public text and image datasets demonstrate that ARTL significantly outperforms state‑of‑the‑art transfer learning methods.

Abstract

Domain transfer learning, which learns a target classifier using labeled data from a different distribution, has shown promising value in knowledge discovery yet still been a challenging problem. Most previous works designed adaptive classifiers by exploring two learning strategies independently: distribution adaptation and label propagation. In this paper, we propose a novel transfer learning framework, referred to as Adaptation Regularization based Transfer Learning (ARTL), to model them in a unified way based on the structural risk minimization principle and the regularization theory. Specifically, ARTL learns the adaptive classifier by simultaneously optimizing the structural risk functional, the joint distribution matching between domains, and the manifold consistency underlying marginal distribution. Based on the framework, we propose two novel methods using Regularized Least Squares (RLS) and Support Vector Machines (SVMs), respectively, and use the Representer theorem in reproducing kernel Hilbert space to derive corresponding solutions. Comprehensive experiments verify that ARTL can significantly outperform state-of-the-art learning methods on several public text and image datasets.