Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines

TLDR

Continual learning has attracted significant attention, yet diverse evaluation scenarios hinder meaningful comparison. This work systematically categorizes continual learning scenarios and evaluates them within a consistent framework. The authors benchmark state‑of‑the‑art methods and strong baselines across the defined scenarios. Results show that simple baselines such as Adagrad, L2 regularization, and naive rehearsal can match mainstream methods, revealing relative scenario difficulty. Code is available at https://github.com/GT-RIPL/Continual-Learning-Benchmark.

Abstract

Continual learning has received a great deal of attention recently with several approaches being proposed. However, evaluations involve a diverse set of scenarios making meaningful comparison difficult. This work provides a systematic categorization of the scenarios and evaluates them within a consistent framework including strong baselines and state-of-the-art methods. The results provide an understanding of the relative difficulty of the scenarios and that simple baselines (Adagrad, L2 regularization, and naive rehearsal strategies) can surprisingly achieve similar performance to current mainstream methods. We conclude with several suggestions for creating harder evaluation scenarios and future research directions. The code is available at https://github.com/GT-RIPL/Continual-Learning-Benchmark