Continual Attentive Fusion for Incremental Learning in Semantic Segmentation

Abstract

Over the past years, semantic segmentation, as many other tasks in computer\nvision, benefited from the progress in deep neural networks, resulting in\nsignificantly improved performance. However, deep architectures trained with\ngradient-based techniques suffer from catastrophic forgetting, which is the\ntendency to forget previously learned knowledge while learning new tasks.\nAiming at devising strategies to counteract this effect, incremental learning\napproaches have gained popularity over the past years. However, the first\nincremental learning methods for semantic segmentation appeared only recently.\nWhile effective, these approaches do not account for a crucial aspect in\npixel-level dense prediction problems, i.e. the role of attention mechanisms.\nTo fill this gap, in this paper we introduce a novel attentive feature\ndistillation approach to mitigate catastrophic forgetting while accounting for\nsemantic spatial- and channel-level dependencies. Furthermore, we propose a\n{continual attentive fusion} structure, which takes advantage of the attention\nlearned from the new and the old tasks while learning features for the new\ntask. Finally, we also introduce a novel strategy to account for the background\nclass in the distillation loss, thus preventing biased predictions. We\ndemonstrate the effectiveness of our approach with an extensive evaluation on\nPascal-VOC 2012 and ADE20K, setting a new state of the art.\n