Bayesian Model-Agnostic Meta-Learning

TLDR

Learning to infer Bayesian posterior from a few‑shot dataset is an important step toward robust meta‑learning due to the model uncertainty inherent in the problem. The paper proposes a novel Bayesian model‑agnostic meta‑learning method. It combines scalable gradient‑based meta‑learning with nonparametric variational inference, introduces a robust Bayesian meta‑update with a new meta‑loss to prevent overfitting, and remains an efficient, model‑agnostic, and simple‑to‑implement gradient‑based learner. During fast adaptation it learns complex uncertainty structures beyond point estimates or simple Gaussians, and experiments demonstrate its accuracy and robustness across sinusoidal regression, image classification, active learning, and reinforcement learning tasks.

Abstract

Learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning due to the model uncertainty inherent in the problem. In this paper, we propose a novel Bayesian model-agnostic meta-learning method. The proposed method combines scalable gradient-based meta-learning with nonparametric variational inference in a principled probabilistic framework. During fast adaptation, the method is capable of learning complex uncertainty structure beyond a point estimate or a simple Gaussian approximation. In addition, a robust Bayesian meta-update mechanism with a new meta-loss prevents overfitting during meta-update. Remaining an efficient gradient-based meta-learner, the method is also model-agnostic and simple to implement. Experiment results show the accuracy and robustness of the proposed method in various tasks: sinusoidal regression, image classification, active learning, and reinforcement learning.