IEEE Transactions on Pattern Analysis and Machine Intelligence · 2023 · 54 citations · 24 references
ComputingComputer ScienceMachine LearningStatistical LearningTraining SystemsAdaptive LearningDeep LearningMeta-learningFew-shot LearningLearningMeta-initializationMeta-trainingModel-agnostic Meta-learningRapid AdaptationTask AdaptationWeight Decay
The objective of few-shot learning is to design a system that can adapt to a given task with only few examples while achieving generalization. Model-agnostic meta-learning (MAML), which has recently gained the popularity for its simplicity and flexibility, learns a good initialization for fast adaptation to a task under few-data regime. However, its performance has been relatively limited especially when novel tasks are different from tasks previously seen during training. In this work, instead of searching for a better initialization, we focus on designing a better fast adaptation process. Consequently, we propose a new task-adaptive weight update rule that greatly enhances the fast adaptation process. Specifically, we introduce a small meta-network that can generate per-step hyperparameters for each given task: learning rate and weight decay coefficients. The experimental results validate that learning a good weight update rule for fast adaptation is the equally important component that has drawn relatively less attention in the recent few-shot learning approaches. Surprisingly, fast adaptation from random initialization with ALFA can already outperform MAML. Furthermore, the proposed weight-update rule is shown to consistently improve the task-adaptation capability of MAML across diverse problem domains: few-shot classification, cross-domain few-shot classification, regression, visual tracking, and video frame interpolation.
24
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su et al. · International Journal of Computer Vision · 2015 · 40.2K citations
Caltech-UCSD Birds-200-2011 Dataset
Catherine Wah · CaltechAUTHORS (California Institute of Technology) · 2024 · 3.9K citations · Full text
Learning Multi-domain Convolutional Neural Networks for Visual Tracking
Hyeonseob Nam, Bohyung Han · 2016 · 2.8K citations
A Database and Evaluation Methodology for Optical Flow
Simon Baker, Daniel Scharstein, John Lewis et al. · International Journal of Computer Vision · 2010 · 2.2K citations · Full text