Learning to Prompt for Continual Learning

TLDR

Continual learning traditionally tackles catastrophic forgetting by adapting model parameters to non‑stationary data distributions, often relying on rehearsal buffers or task identities. The study proposes a novel continual learning paradigm that trains a compact memory system without needing task identities at test time. The authors introduce L2P, a method that learns small, dynamic prompts for a pre‑trained model, optimizing them to guide predictions and balance task‑invariant and task‑specific knowledge while preserving plasticity. Experiments on popular image‑classification benchmarks show that L2P consistently outperforms state‑of‑the‑art methods and rivals rehearsal‑based approaches even without a buffer, proving effective for task‑agnostic continual learning. Source code is available at https://github.com/google-research/12p.

Abstract

The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. Typical methods rely on a rehearsal buffer or known task identity at test time to retrieve learned knowl-edge and address forgetting, while this work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequen-tially under different task transitions. In our proposed framework, prompts are small learnable parameters, which are maintained in a memory space. The objective is to optimize prompts to instruct the model prediction and ex-plicitly manage task-invariant and task-specific knowledge while maintaining model plasticity. We conduct comprehen-sive experiments under popular image classification bench-marks with different challenging continual learning set-tings, where L2P consistently outperforms prior state-of-the-art methods. Surprisingly, L2P achieves competitive results against rehearsal-based methods even without a re-hearsal buffer and is directly applicable to challenging task-agnostic continual learning. Source code is available at https://github.com/google-research/12p.