Proceedings of the AAAI Conference on Artificial Intelligence · 2021 · 21 citations · 20 references
ComputingComputer ScienceArtificial IntelligenceMachine LearningNatural Language ProcessingDeep LearningExplainable Machine LearningExplanationNeural NetworksPattern RecognitionRepresentation LearningConstraintsExplainable Artificial IntelligenceExplanation RobustnessModel InterpretabilityTrainingTraining TimeUser-centric Explanations
Explaining black-box models such as deep neural networks is becoming increasingly important as it helps to boost trust and debugging. Popular forms of explanations map the features to a vector indicating their individual importance to a decision on the instance-level. They can then be used to prevent the model from learning the wrong bias in data possibly due to ambiguity. For instance, Ross et al.'s ``right for the right reasons'' propagates user explanations backwards to the network by formulating differentiable constraints based on input gradients. Unfortunately, input gradients as well as many other widely used explanation methods form an approximation of the decision boundary and assume the underlying model to be fixed. Here, we demonstrate how to make use of influence functions---a well known robust statistic---in the constraints to correct the model’s behaviour more effectively. Our empirical evidence demonstrates that this ``right for better reasons''(RBR) considerably reduces the time to correct the classifier at training time and boosts the quality of explanations at inference time compared to input gradients. Besides, we also showcase the effectiveness of RBR in correcting "Clever Hans"-like behaviour in real, high-dimensional domain.
20
Explaining and Harnessing Adversarial Examples
Ian Goodfellow, Jonathon Shlens, Christian Szegedy · arXiv (Cornell University) · 2014 · 8.2K citations · Full text
Dissecting racial bias in an algorithm used to manage the health of populations
Ziad Obermeyer, Brian W. Powers, Christine Vogeli et al. · Science · 2019 · 6.1K citations · Full text
Public Health, Health Care Management, Health Informatics +17