Proceedings of the AAAI Conference on Artificial Intelligence · 2020 · 88 citations · 28 references
ComputingComputer ScienceComputer VisionMachine LearningNatural Language ProcessingDeep LearningNeural NetworksRepresentation LearningEvaluation MetricsSegmentationSegmentation Evaluation Metrics
Deep Neural Networks have achieved superhuman performance across many domains, prompting a growing need to understand their complex internal mechanisms. This paper introduces Relative Attributing Propagation (RAP), a method that decomposes DNN predictions by separating positive and negative attributions based on relative influence between layers. RAP assigns each neuron a bi‑polar importance score by distinguishing positive and negative relevance while preserving conservation, and its validity is assessed using outside‑inside relevance ratio, segmentation mIOU, and region perturbation metrics. Across all experiments, RAP produces clearer, more attentive visualizations of separated attributions and demonstrates a sizable performance gap over existing methods.
As Deep Neural Networks (DNNs) have demonstrated superhuman performance in a variety of fields, there is an increasing interest in understanding the complex internal mechanisms of DNNs. In this paper, we propose Relative Attributing Propagation (RAP), which decomposes the output predictions of DNNs with a new perspective of separating the relevant (positive) and irrelevant (negative) attributions according to the relative influence between the layers. The relevance of each neuron is identified with respect to its degree of contribution, separated into positive and negative, while preserving the conservation rule. Considering the relevance assigned to neurons in terms of relative priority, RAP allows each neuron to be assigned with a bi-polar importance score concerning the output: from highly relevant to highly irrelevant. Therefore, our method makes it possible to interpret DNNs with much clearer and attentive visualizations of the separated attributions than the conventional explaining methods. To verify that the attributions propagated by RAP correctly account for each meaning, we utilize the evaluation metrics: (i) Outside-inside relevance ratio, (ii) Segmentation mIOU and (iii) Region perturbation. In all experiments and metrics, we present a sizable gap in comparison to the existing literature.
28
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su et al. · International Journal of Computer Vision · 2015 · 40.2K citations
Learning Deep Features for Discriminative Localization
Bolei Zhou, Aditya Khosla, Àgata Lapedriza et al. · 2016 · 10.8K citations · Full text
A Unified Approach to Interpreting Model Predictions
Scott Lundberg, Su‐In Lee · arXiv (Cornell University) · 2017 · 7.6K citations · Full text
The Pascal Visual Object Classes Challenge: A Retrospective
Mark Everingham, S. M. Ali Eslami, Luc Van Gool et al. · International Journal of Computer Vision · 2014 · 7.3K citations · Full text