2017 · 209 citations · 33 references
ComputingComputer ScienceArtificial IntelligenceComputer VisionMachine LearningContinual LearningDeep LearningNeural NetworksPattern RecognitionRepresentation LearningSupervised LearningClass-incremental LearningDeep Learning-based ClassificationIncremental LearningProgressive Neural Networks
Incremental learning systems that acquire new concepts over time from streaming data remain a major open problem in AI, especially since prior approaches were limited to fixed representations incompatible with deep learning. This work introduces iCaRL, a training strategy that enables class‑incremental learning with only a small subset of classes present at any time, allowing new classes to be added progressively. iCaRL jointly learns robust classifiers and a data representation. Experiments on CIFAR‑100 and ImageNet demonstrate that iCaRL can learn many classes incrementally over a long period, outperforming other strategies that quickly fail.
A major open problem on the road to artificial intelligence is the development of incrementally learning systems that learn about more and more concepts over time from a stream of data. In this work, we introduce a new training strategy, iCaRL, that allows learning in such a class-incremental way: only the training data for a small number of classes has to be present at the same time and new classes can be added progressively. iCaRL learns strong classifiers and a data representation simultaneously. This distinguishes it from earlier works that were fundamentally limited to fixed data representations and therefore incompatible with deep learning architectures. We show by experiments on CIFAR-100 and ImageNet ILSVRC 2012 data that iCaRL can learn many classes incrementally over a long period of time where other strategies quickly fail.
33
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 58.2K citations · Full text
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su et al. · International Journal of Computer Vision · 2015 · 40.2K citations
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky et al. · 2014 · 34.3K citations