Generalizing from a Few Examples: A Survey on Few-Shot Learning

TLDR

Machine learning excels with large datasets but struggles when data are scarce, prompting the development of Few‑Shot Learning to address small‑sample scenarios. This survey aims to comprehensively understand Few‑Shot Learning and outline promising future directions across problem setups, techniques, applications, and theory. The authors formalize Few‑Shot Learning, distinguish it from related problems, and classify methods into data‑augmentation, model‑reduction, and algorithm‑search categories based on how prior knowledge is leveraged. They identify unreliable empirical risk as the core challenge in Few‑Shot Learning and use the taxonomy to evaluate each category’s strengths and weaknesses.

Abstract

Machine learning has been highly successful in data-intensive applications but is often hampered when the data set is small. Recently, Few-Shot Learning (FSL) is proposed to tackle this problem. Using prior knowledge, FSL can rapidly generalize to new tasks containing only a few samples with supervised information. In this paper, we conduct a thorough survey to fully understand FSL. Starting from a formal definition of FSL, we distinguish FSL from several relevant machine learning problems. We then point out that the core issue in FSL is that the empirical risk minimized is unreliable. Based on how prior knowledge can be used to handle this core issue, we categorize FSL methods from three perspectives: (i) data, which uses prior knowledge to augment the supervised experience; (ii) model, which uses prior knowledge to reduce the size of the hypothesis space; and (iii) algorithm, which uses prior knowledge to alter the search for the best hypothesis in the given hypothesis space. With this taxonomy, we review and discuss the pros and cons of each category. Promising directions, in the aspects of the FSL problem setups, techniques, applications and theories, are also proposed to provide insights for future research.