Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · 2022 · 11 citations · 19 references
ComputingComputer ScienceFeature EngineeringMachine LearningNatural Language ProcessingDeep LearningRepresentation LearningAdaptation To New EnvironmentsDomain AdaptationFeature NormalizationFew-shot Domain AdaptationFew-shot LearningFew-shot Semantic SegmentationMini-batch TrainingPerformance Degradation
Deep networks are prone to performance degradation when there is a domain shift between the source (training) data and target (test) data. Recent test-time adaptation methods update batch normalization layers of pre-trained source models deployed in new target environments with streaming data. Although these methods can adapt on-the-fly without first collecting a large target domain dataset, their performance is dependent on streaming conditions such as mini-batch size and class-distribution which can be unpredictable in practice. In this work, we propose a framework for few-shot domain adaptation to address the practical challenges of data-efficient adaptation. Specifically, we propose a constrained optimization of feature normalization statistics in pre-trained source models supervised by a small target domain support set. Our method is easy to implement and improves source model performance with as little as one sample per class for classification tasks. Extensive experiments on 5 cross-domain classification and 4 semantic segmentation datasets show that our proposed method achieves more accurate and reliable performance than test-time adaptation, while not being constrained by streaming conditions.
19
Sinno Jialin Pan, Qiang Yang · IEEE Transactions on Knowledge and Data Engineering · 2009 · 23.2K citations
Model Adaptation: Unsupervised Domain Adaptation Without Source Data
Rui Li, Qianfen Jiao, Wenming Cao et al. · 2020 · 494 citations · Full text
Domain-Specific Batch Normalization for Unsupervised Domain Adaptation
Woong-Gi Chang, Tackgeun You, Seonguk Seo et al. · 2019 · 434 citations
Revisiting Batch Normalization For Practical Domain Adaptation
Yanghao Li, Naiyan Wang, Jianping Shi et al. · arXiv (Cornell University) · 2016 · 372 citations · Full text