2019 · 13 citations · 7 references
Social SciencesPsychologyAgingKnowledge RetentionMemoryRecall MemorySynaptic PruningCatastrophic ForgettingLearningNeural PruningPruningPruning Of Neural ConnectionsResource Limitations
Grow-when-required networks such as the Growing Dual-Memory (GDM) networks possess a dynamic network structure, expanding to accommodate new neurons in response to learning novel concepts. Over time, it may be necessary to prune obsolete neurons and/or neural connections to meet performance or resource limitations. GDM networks utilize an age-based pruning strategy, whereby older neurons and neural connections that have not been activated recently are removed. Catastrophic forgetting occurs when knowledge learned by the networks in previous learning iterations is lost due to being overwritten by newer learning iterations, or to the pruning process. In this work, we investigate catastrophic forgetting in GDM networks in response to different pruning strategies. The age-based pruning method was shown to significantly sparsify the GDM network topology while improving the networks ability to recall newly acquired concepts with a slight decrease in performance with respect to older knowledge. A significance-based pruning method was tested as a replacement for the age-based pruning, but was not as effective at pruning even though it performed better at recalling older knowledge.
7
Measuring Catastrophic Forgetting in Neural Networks
Ronald Kemker, Marc McClure, Angelina Abitino et al. · arXiv (Cornell University) · 2017 · 191 citations · Full text
Lifelong learning of human actions with deep neural network self-organization
German I. Parisi, Jun Tani, Cornelius Weber et al. · Neural Networks · 2017 · 134 citations · Full text