2022 · 87 citations · 30 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingActive LearningLanguage ModelingLanguage ModelsLearning From DemonstrationPolicy LearningReinforcement LearningActive Learning From DemonstrationDemonstration-based Policy LearningIn-context LearningLanguage Processing ModelsLarge Language ModelsLarge-scale Natural Language Processing
Large‑scale language models can perform various tasks by in‑context learning from a handful of demonstration examples without fine‑tuning. We formulate example selection for in‑context learning as a sequential decision problem. We propose a reinforcement learning algorithm to identify generalizable policies for selecting demonstration examples. We show that in‑context learning performance is highly unstable across example samples, but our reinforcement‑learning–based example selection yields a 5.8 % average improvement on GPT‑2 and modest gains on GPT‑3 Ada, while the benefit decreases on larger GPT‑3 models, indicating emerging capabilities of large language models.
With a handful of demonstration examples, large-scale language models demonstrate strong capability to perform various tasks by in-context learning from these examples, without any fine-tuning. We demonstrate that in-context learning performance can be highly unstable across samples of examples, indicating the idiosyncrasies of how language models acquire information. We formulate example selection for in-context learning as a sequential decision problem, and propose a reinforcement learning algorithm for identifying generalizable policies to select demonstration examples. For GPT-2, our learned policies demonstrate strong abilities of generalizing to unseen tasks in training, with a 5.8% improvement on average. Examples selected from our learned policies can even achieve a small improvement on GPT-3 Ada. However, the improvement diminishes on larger GPT-3 models, suggesting emerging capabilities of large language models.
30
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Richard Socher, Alex Perelygin, Jean Y. Wu et al. · 2013 · 6.8K citations · Full text
Active Learning Literature Survey
Burr Settles · Minds at UW (University of Wisconsin) · 2009 · 4.8K citations · Full text