Online Fast Adaptation and Knowledge Accumulation: a New Approach to\n Continual Learning

Abstract

Continual learning studies agents that learn from streams of tasks without\nforgetting previous ones while adapting to new ones. Two recent\ncontinual-learning scenarios have opened new avenues of research. In\nmeta-continual learning, the model is pre-trained to minimize catastrophic\nforgetting of previous tasks. In continual-meta learning, the aim is to train\nagents for faster remembering of previous tasks through adaptation. In their\noriginal formulations, both methods have limitations. We stand on their\nshoulders to propose a more general scenario, OSAKA, where an agent must\nquickly solve new (out-of-distribution) tasks, while also requiring fast\nremembering. We show that current continual learning, meta-learning,\nmeta-continual learning, and continual-meta learning techniques fail in this\nnew scenario. We propose Continual-MAML, an online extension of the popular\nMAML algorithm as a strong baseline for this scenario. We empirically show that\nContinual-MAML is better suited to the new scenario than the aforementioned\nmethodologies, as well as standard continual learning and meta-learning\napproaches.\n