Artificial Intelligence · 2022 · 11 citations · 39 references
ComputingData ScienceMachine LearningPredictive AnalyticsProbability TheoryExplainable Machine LearningModel ComparisonProbabilistic Machine LearningStatistical InferenceInteraction EffectsInteraction Effects ModelingInteraction MeasureModel-agnostic ExplanationPredictionPrediction TheoryShapley OperatorShapley Value Attribution
PredDiff is a model-agnostic, local attribution method that is firmly rooted in probability theory. Its simple intuition is to measure prediction changes while marginalizing features. In this work, we clarify properties of PredDiff and its close connection to Shapley values. We stress important differences between classification and regression, which require a specific treatment within both formalisms. We extend PredDiff by introducing a new, well-founded measure for interaction effects between arbitrary feature subsets. The study of interaction effects represents an inevitable step towards a comprehensive understanding of black-box models and is particularly important for science applications. Equipped with our novel interaction measure, PredDiff is a promising model-agnostic approach for obtaining reliable, numerically inexpensive and theoretically sound attributions.
39
Advances in Neural Information Processing Systems 29
Onur Teymur, Kostas Zygalakis, Ben Calderhead · 2016 · 13.5K citations
From local explanations to global understanding with explainable AI for trees
Scott Lundberg, Gabriel Erion, Hugh Chen et al. · Nature Machine Intelligence · 2020 · 8.9K citations · Full text