2019 · 46 citations · 9 references
ComputingComputer ScienceArtificial IntelligenceMachine LearningMemoryContinual LearningDeep LearningNeural NetworksCatastrophic ForgettingData DistributionFeedforward Neural NetworksRegularizationRegularization TechniquesRegularization-based Continual Learning
Continual learning broadly refers to the algorithms which aim to learn continuously over time across varying domains, tasks or data distributions. This is in contrast to algorithms restricted to learning a fixed number of tasks in a given domain, assuming a static data distribution. In this survey we aim to discuss a wide breadth of challenges faced in a continual learning setup and review existing work in the area. We discuss parameter regularization techniques to avoid catastrophic forgetting in neural networks followed by memory based approaches and the role of generative models in assisting continual learning algorithms. We discuss how dynamic neural networks assist continual learning by endowing neural networks with a new capacity to learn further. We conclude by discussing possible future directions.
9
Continual Learning Through Synaptic Intelligence.
Friedemann Zenke, Ben Poole, Surya Ganguli · PubMed · 2017 · 979 citations · Full text
Computing, Biology, Computer Science +13
Continual Learning with Deep Generative Replay
Hanul Shin, Jung Kwon Lee, Jaehong Kim et al. · arXiv (Cornell University) · 2017 · 202 citations · Full text